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Preface 


The 6th Hamlyn Symposium on Medical Robotics was held at The Hamlyn Centre for Robotic 
Surgery and The Royal Academy of Engineering in London, UK on 22-25 th June, 2013. 


The theme for this year’s Symposium was “co-operative control and perceptual docking ” and there 
was an impressive line-up of leading scientists and engineers in collaborative robotics, navigation and 
image guidance, in particular, as invited Speakers. The Speakers invited included Professor Robert 
Howe and Dr Reiza Rayman, who both provided insights into the cutting edge developments in this 
field. The keynote was delivered by Gary Guthart, who provided an overview of “The daVinci System 
at 14: Clinical OverView, Economics and Opportunities” . This year’s annual Karl Storz-Harold 
Hopkins lecture was delivered by Professor Ferdinand Köckerling on 44 Robotics in Bariatric Surgery ”. 


A total of 102 papers were submitted from 18 countries and after systematic peer review, 62 papers 
were selected for presentation at the Symposium. The topics covered ranged from training and clinical 
outcomes, image guidance in robotic surgery, platform design, perceptual docking, intraoperative 
tissue tracking and characterisation, and new clinical approaches and pilot studies. 


The Hamlyn Symposium grew out of the original Imperial College’s Cross Faculty Workshops on 
Medical Robotics funded by the Hamlyn Centre for Robotic Surgery. We are delighted that it is now 
established as an annual international forum for clinicians, engineers and researchers to exchange 
ideas and explore new challenges and opportunities in healthcare technologies. We were encouraged 
to see that this year senior academics and young research fellows had continued to work together to 
organise a series of workshops on topics including Assistive Robots, Flexible Access Surgery, 
Regulatory Affairs and Clinical Trials and Outcomes, held before and after the Symposium. These 
workshops provided a forum for focused discussion about specific research and clinical topics of 
medical robotics, as well as an informal environment for academic networking. Many of these 
workshops have resulted in further on-going interactions among the participants and joint research 
publications. 


We would like to thank both the International and Local Programme Committees, the Workshop 
Organising Committee and the Local Organising Committee for giving up their precious time 
ensuring timely review of all the papers submitted and helping to provide an excellent Symposium 
Programme. The meeting would not be possible without the commitment and hard work of a 
dedicated team. In particular, we are grateful to Karen Kerr, Su-Lin Lee, Hedyeh Rafii-Tari, George 
Mylonas, Daniel Elson, Sejal Jiwan, Ruzanna Gulakyan, and Raphaele Raupp for their tireless effort 
in managing all aspects of the Symposium Organisation. Last but not least, we are grateful to all the 
workshop organisers for making all the pre- and post-symposium works a huge success. A photo 
album is available online (http://www.hamlyn-robotics.org/), which captures the memorable 
experience of the entire event. 


We would like to take this opportunity to give our special thanks to Lady Hamlyn. All of this would, 
of course, not be possible without the generous Philanthropie Support from both the Helen Hamlyn 
Trust and Lady Hamlyn herseif. 


It was our pleasure to welcome the Symposium attendees to London. 


June 2013, London 


Guang-Zhong Yang, FREng 
Ara Darzi, FRS 
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Karl Storz-Harold Hopkins Lecture 
Robotic Technologies in Bariatric Surgery 

F. Köckerling, V. Lange 

Department of Surgery and Center for Minimally Invasive Surgery 
Academic Teaching Hospital of Charite Medical School 

Vivantes Hospital Berlin 

Ferdinand. koeckerling@ vivantes. de 


INTRODUCTION 

Robotic surgery has undergone rapid development in 
recent years. In 2009 around 205,000 robot-assisted 
procedures were performed worldwide. In the USA 
there are approximately 1,400 daVinci Systems in use, 
and 400 in the remainder of the world. One major 
disadvantage of the daVinci procedures is the 
additional variable costs of 1 ,600 dollars per procedure, 
rising to 3,200 dollars per Operation when the 
amortization costs are included. 

Equally fast-paced has been the ongoing development 
of bariatric surgery over the past two decades. 

The gold Standard in bariatric surgery is the 
laparoscopic Roux-en-Y gastric bypass (RYGB). 
Based on recent reports, some 350, 000 bariatric 
procedures are being performed each year 
worldwide. Laparoscopic RYGB is arguably one of the 
most challenging minimally invasive procedures in 
general surgery, with a steep learning curve of 75 to 
1 00 cases to achieve the highest level of proficiency. It 
is therefore understandable that studies are being 
carried out to investigate the use of robotic 
technologies to facilitate conduct of bariatric surgery. 
However, when assessing the use of robotic 
technologies in bariatric surgery, a distinction must be 
made between conduct of the laparoscopic RYGB 
procedure with the daVinci System and implementation 
of the various steps of a laparoscopic RYGB Operation 
with robotic instruments and devices that make the 
Operation swifter and safer. 

MATERIALS AND METHODS 

To date, no randomized comparative studies or meta- 
analyses have been carried out to investigate 
differences between the robot-assisted DaVinci-RYGB 
and the classic laparoscopic RYGB. There are only 
systematic reviews with summaries of the published 
case series. 

RESULTS 

A systematic review by Gill et al (2011) with 1,253 
patients with a mean BMI of 46.6 identified an 
anastomotic leak rate of 2.4%, bleeding rate of 2% and 
Stenosis in 3% of cases [1]. The mortality rate was 0%. 
Another systematic review by Fourman et al (2012) 
found similar or lower complication rates compared 


with traditional laparoscopy [2]. Two studies showed 
shorter operative times using the robot, but 4 studies 
showed longer operative times. The learning curve 
appears to be shorter. A further systematic review with 
7 studies and 1,686 patients found a significantly 
reduced incidence of anastomotic strictures [3]. This 
study showed no difference in anastomotic leaks, post- 
operative complications, operative time or length of 
hospital stay. The largest published series with 1,100 
patients from two high- volume centers reported a mean 
Operation time of 155 minutes, an anastomotic leak rate 
of 0.09% and a zero mortality rate [4]. These 
impressive data clearly show that the gold Standard in 
bariatric surgery, the minimally invasive Roux-en-Y 
gastric bypass, can be routinely performed with the 
daVinci robot with excellent results. The learning 
curve seems to be shorter than that for the traditional 
laparoscopic RYGB. 

To ans wer the question, with a high evidence level, of 
whether the Rouy-en-Y gastric bypass performed with 
the daVinci robot is better than laparoscopic gastric 
bypass, larger prospective randomized trials and meta- 
analyses are needed. Since the differences in outcome 
are apparently not too great, these studies need 
correspondingly larger case numbers. 

In addition to robotic surgery with the Vinci System, 
bariatric surgery robotic technologies have been 
introduced to simplify and optimize the individual 
operative steps. These include robotic trocars, optical 
access Systems to visualize first trocar insertion, self- 
retaining retract robots, such as the Nathanson liver 
retractor, and robotic stapling Systems. There are 
important considerations between tissue and device 
interaction that can affect staple line outcomes. 
Stapling technologies are being developed which, via a 
sensor, are able to determine tissue thickness and, 
accordingly, apply the appropriate staple height. 
Likewise, the temporal course of tissue dissection 
between the rows of clip sutures is controlled by the 
device so as to assure a more precise tissue incision. 

DISCUSSION 

The deployment of such robotic technologies for the 
individual critical RYGB steps and for simplification 
of the classic laparoscopic procedure represents a more 
cost-effective alternative to the daVinci Operation, 
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something that plays an important role in healthcare 
Systems where payments are calculated on the basis of 
Diagnosis Related Groups (DRGs). 

REFERENCES 

[1] Gill RS, Al-Adra DP, Birch D, Hudson M, Shi X, 
Sharma AM, Karmali S. Robotic-assisted bariatric 
surgery: a systematic review. The International Journal 
of Medical Robotics and Computer Assisted Surgery 
2011,7:249-255 

[2] Fourman MM, Saber AA. Robotic bariatric surgery: a 
systematic review. Surgery for Obesity and Related 
Diseases 2012, 8:483-488 


[3] Markar SR, Karthikesalingam AP, Venkat-Ramen V, 
Kinross J, Ziprin P. Robotic vs. laparoscopiec Roux-en- 
Y gastric bypass in morbidly obese patient: systematic 
review and pooled analysis. The International Journal 
of Medical Robotics and Computer Assisted Surgery 
2011,7:393-400 

[4] Tieu K, Allison N, Snyder B, Wilson T, Toder M, 
Wilson E. Robotic-assisted Roux-en-Y gastric bypass: 
update from 2 high- volume centers. Surgery for 
Obesity and Related Diseases 2013, 9:284-289 


Every year at the Hamlyn Symposium, an invitation is extended to an outstanding researcher in the field of Minimally 
Invasive Surgery to present the Karl Storz-Harold Hopkins Lecture. Harold Hopkins (1918-1994) was a renowned 
British physicist whose work on optical physics would be used in television, laser-disc and CD players, and, of course, 
endoscopy. At the time, existing scopes had poor image transmission and Hopkins’ introduction of a glass rod lens to 
these rigid scopes solved many of these problems, resulting in the creation of a Hopkins rod lens scope. When Karl 
Storz leamed about the Hopkins scope, he agreed to invest in it and added a ‘cold light’ fibre optic application for 
illumination - a design that has not changed significantly today. Düring his career, Harold Hopkins was presented with 
many awards, not least the Fellowship of the Royal Society and Honorary Fellow of the Royal College of Surgeons. 
Endoscopy in every field of medicine owes its present state to this very remarkable man and we are honoured to host 
this annual lecture, in partnership with Karl Storz Ltd. 
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Design of a Bone-Attached Robot for Mastoidectomy 

N.P. Dillon 1 , R.J. Webster III 1 ’ 2 , T.J. Withrow 1 

Department of Mechanical Engineering, Vanderbilt University 
2 Department of Otolaryngology, Vanderbilt University Medical Center 

neal.p. dillon@vanderbilt. edu 


INTRODUCTION 

The mastoidectomy is a common otologic procedure 
performed as a component of other surgeries such as 
cochlear implantation and acoustic neuroma tumor 
removal as well as to treat infections in the mastoid. 
Sub-millimetric accuracy is required to avoid damage to 
critical anatomic structures embedded in bone such as 
the facial nerve, sigmoid sinus, external auditory canal, 
and carotid artery [1]. This accuracy requirement and 
the fact that many of the critical structures are hidden 
within the bone, makes it a strong candidate for image- 
guided robotic surgery. 

Several research groups have investigated the 
feasibility of robotic mastoidectomy using a serial, 
industrial robot with a surgical drill attached to its end- 
effector (e.g. [2-3]), an approach similar to the 
commercial Mako RIO robot typically used for joint 
repair/replacement surgeries (Mako Surgical Corp., Ft. 
Lauderdale, FL, USA). Using this approach, patient 
motion must be monitored so that the drill tip remains 
correctly registered with the target anatomy at all times. 
This registration process introduces an additional 
uncertainty in the Overall System, and requires 
cumbersome error checking to prevent failure of the 
patient motion monitoring System. To mitigate this 
added uncertainty and reduce error checking bürden, we 
propose to use a bone-attached robot. 

While such a System may be beneficial for 
registering the robot to the patient anatomy, it also 
presents an additional set of design challenges. The size 
constraint of a robot that can be attached directly to the 
patient restricts the power of the actuators and the 
stiffness of the entire robot. The workspace of the robot 
is also limited with the bone-attached approach. Along 
with the general requirements of traditional 
mastoidectomy, these constraints must be accounted for 
in the design of a bone-attached System. This paper 
outlines several of these design considerations and 
presents the first Version of the robot. 

MATERIALS AND METHODS 

The forces required for robotic mastoidectomy were 
evaluated experimentally by milling formalin-fixed 
temporal bone specimens using an industrial robot 
under various cutting conditions. A Standard otologic 
drill and cutting burrs were used and the experiments 
tested different cutting burr sizes and types, drill angles, 
cutting depths and velocities, and different bone types 
(cortical/surface bone and pneumatized bone of the 
mastoid). The full details of this study are given in [4]. 


The required workspace for a mastoidectomy robot 
was evaluated in terms of overall size and the angular 
requirements of the drill shaft so that all target points 
are safely reached without removing any unnecessary 
bone or violating critical anatomy. Ten temporal bone 
specimens were used in this study. The specimens were 
formalin-fixed bones previously used by residents in the 
department of Otolaryngology at Vanderbilt University 
Medical Center for training purposes. The set of ten 
was selected by an experienced surgeon who verified 
the mastoidectomy procedures of each and that the set 
of bones were representative of a broad ränge of patient 
anatomy. The bones were scanned using a Xoran xCAT 
ENT portable CT Scanner (Xoran Technologies, Ine., 
Ann Arbor, MI, USA). The removed volume of bone 
for each mastoidectomy was segmented using custom 
Software with a semi-automated approach. The 
outermost surfaces of the removed volume for each 
specimen (i.e. the open face of the cavity) were aligned 
since this is the location where a bone-attached robot 
would attach to the patient. An overall workspace size 
was calculated from this aligned set of specimens. 


X-Axis (mm) 

Fig. 1 One safe and one unsafe drill orientation for a given 
target point. In the unsafe case, the drill shaft passes through 
bone outside of the target volume. 

The angular workspace was calculated by examining 
the allowable drill angles for each specimen. The 
removed volume of bone was discretized into a series of 
target points (1 point for each voxel of the 0.4 mm x 0.4 
mm x 0.4 mm scan) and each individual point was 
checked to determine the angles required to reach the 
point without the drill Crossing out of the target volume 
at any point along its shaft (see Figure 1). Two cases 
were examined: 4 total degrees-of-freedom (1 angular 
DOF) and 5 total DOF (2 angular DOF). Figure 2 
shows the orientation angles of the drill shaft relative to 
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the top surface of the target volume (lateral surface of 
skull where the robot is attached). In the 4 DOF case it 
was assumed that the robot could be attached to the 
patient at various Orientations of its base (cp angle); 
however, this angle is fixed once the robot is attached. 
In the 5 DOF case, the drill was free to rotate about cp as 
well as 0. For each specimen, the ränge of angles 
required for the target points to be safely reached was 
calculated. 


Plane of rutation of 0 



Fig. 2 Drill orientation angles relative to the patient. 

RESULTS 

The mean forces for various cutting conditions ranged 
from 0.4 N to 3.4 N with maximum force Spikes up to 
13.1 N. It was determined that larger burrs can remove 
a specified volume of bone in a given time with lower 
forces than smaller burrs and utilizing the side of the 
spherical burr (rather than the distal tip) by adjusting the 
drill angle relative to the bone surface reduced the 
transient force Spikes. Lower forces were observed in 
pneumatized mastoid bone versus cortical/surface bone; 
however, higher Variation was observed in the 
pneumatized bone. Finally, lower forces were observed 
for a given bone removal rate under cutting conditions 
that utilized higher cutting velocities and shallower 
depths rather than slower/deeper cuts. 

The required workspace of the robot was calculated 
to be 41 mm deep with a maximum cross-sectional area 
at the lateral surface of the skull. The workspace at this 
surface can be approximated by an ellipse with major 
diameter of 52 mm and minor diameter of 45 mm. The 
cross-section decreases as the depth into the skull 
increases. In the angular workspace analysis, it was 
determined that at least 98.3% of the target points in 
each specimen could be reached with a 4 DOF robot (7 
of 10 specimens were completely reachable). A drill tilt 
angle ränge of 45° was the maximum ränge required. 
Additionally, in our opinion, the few target points that 
were unreachable were not required to perform the 
surgical procedure. The 5 DOF case allowed for 100% 
of the target points to be reached for each specimen and 
reduced the required tilt angle to 35°. 

The prototype of this robot (see Fig. 3A) utilizes 
four SmarAct piezoelectric linear positioners (SmarAct 
GmbH, Oldenburg, Germany) with a repeatability of 
approximately one micrometer, travel length of 46 mm, 
and maximum velocity of 15 mm/s. Additionally, these 
actuators are available in autoclavable versions, 
allowing for the entire robot to be sterilized without 


being disassembled. Three of the actuators control 
motion in the x-, y- and z-directions, while a fourth 
actuator Controls the tilt in the drill (ränge of -40° to 
+40°) through a rack and pinion and worm-wheel gear 
set. The worm-wheel setup is necessitated by the fact 
that the torque about the shaft Controlling the tilt 
imposes a higher than allowable force along the axis of 
that actuator due to the forces at the drill tip and makes 
this joint non-backdrivable. In a future Version, the rack 
and pinion could be replaced by a rotary positioner. 
The base plate has three attachment points that allow it 
to be positioned at several different Orientations relative 
to the patient through a positioning frame that is 
anchored to the patient and scanned pre-operatively (see 
Fig. 3B). 



Fig. 3 Photograph of robot prototype (A) and positioning 
frame (B) which attaches rigidly to the skull and connects to 
the robot. 

DISCUSSION 

The first prototype for a bone-attached robot for 
mastoidectomy is discussed in this paper along with 
several of the design considerations. A 4 DOF robot 
was chosen to maximize the accuracy and stiffness of 
the robot while keeping the overall robot as small, light 
and stiff as possible. The results of the force 
experiments are important in both robot design and 
motion planning and were combined with a 
mastoidectomy workspace analysis to aid in the design 
process. Future work on this project will include 
verification that the accuracy of this robot meets the 
surgical requirements, motion planning for optimal 
mastoidectomy volume removal, and testing on cadaver 
temporal bone specimens. 
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INTRODUCTION 

Minimally invasive surgical robotics has enabled 
clinicians to operate with reduced trauma due to small 
incisions and long, slender tools. Instrument guidance 
remains a limiting challenge, however, for procedures in 
which image quality is limited and haptic feedback is 
unavailable, e.g., beating-heart intracardiac procedures 
[1]. To address this need, our group has been 
investigating sensing technologies that can envelop a 
robot with a compliant sensing structure and provide 
clinicians with tissue contact and force information over 
its entire surface including its tip. 

Soft sensors are a particularly promising candidate 
technology that, to date, have not been considered for 
medical applications. These sensors are composed of a 
compliant polymer structure containing channels filled 
with a conductive liquid. Under load, the channel cross 
sections deform providing pressure or strain 
measurements measured as changes in resistance [2]. 



Fig. 1 Surgical soft sensors created in our lab. (a) Robotic 
catheter fmgertip sensor measuring contact force and contact 
angle [3]. (b) Tissue contact sensor with diameter of 1.5mm. 
(c) Neuroendoscope pressure-sensing sleeve for monitoring 
contact pressure with parenchymal tissue. (d) Pressure sensing 
laparoscopic port for the Intuitive Surgical da Vinci robot. 

Manufactured using soft lithography, these sensors can 
have a thickness of less than 1 mm, are extremely 
elastic and compliant, can be manufactured with high 
sensor densities and can exhibit high pressure 
sensitivities. Our group has created and evaluated 
several soft sensor morphologies as shown in Fig 1. 
These include tip force and contact angle sensors for 
intracardiac surgery [3], pressure sensing to prevent 
cortical damage for neuroendoscopy, and pressure 
sensing to monitor/prevent ischemia on the laparoscopic 
ports of the Intuitive Surgical da Vinci System. In our 


work, the soft sensors were made sufficiently thin, 
~400pm thick, and sensitive to loads of order 5g. 

The majority of prior work in soft sensors has used 
the conductive liquid metal Eutectic Gallium-Indium 
(eGaln). The properties of eGaln are conducive to soft 
sensing. Its low channel resistance facilitates simple DC 
circuitry; its low viscosity enables easy injection into 
micro-channel geometries and its high surface tension 
reduces bubble formation and facilitates channel filling 
and sealing during manufacture. Unfortunately, eGaln 
is not biocompatible and the development of alternate 
conductive fluids is needed for medical applications. 

In recent work aimed at green chemistry, the use of 
ionic salt Solutions was proposed. The low viscosity of 
aqueous salt Solutions, however, confounds the 
production of bubble-free channels and thus, thickening 
agents are added to facilitate manufacture [4]. Ionic 
Solutions also require AC excitation to prevent ion 
polarization at the electrodes, and a low voltage must be 
used to avoid electrolysis of the fluid, which can cause 
bubble formation and corrosion [4]. This, however, 
proves advantageous since transforming the signal into 
the digital domain increases the signal to noise ratio 
while requiring a relatively simple oscillator circuit to 
monitor channel resistance via oscillation frequency [5]. 

While biocompatible, however, aqueous salt 
Solutions do not produce reliable and robust sensors 
because the gas permeability of the encapsulating 
polymer is high, which allows the water from the 
solution to evaporate [4]. In fact, the permeability of 
siloxane based polymers is so high that they can be used 
to manufacture permeation pumps for micro-fluidic 
Systems [6]. 

The contribution of this paper is to demonstrate that 
NaCl saturated glycerol provides all of the necessary 
properties for medical soft sensors. This solution is 
biocompatible [7], sufficiently conductive for sensing 
applications, adjustably viscous for ease of manufacture 
and, possessing a boiling point over 300°C, it is 
extremely resistant to evaporation. An initial sensor 
design and experimental evaluation is presented below. 

MATERIALS AND METHODS 

To investigate the efficacy of NaCl saturated glycerol as 
a biocompatible conductive fluid, we constructed and 
tested several prototype sensors. NaCl saturated 
glycerol was made by mixing 99% pure glycerol (Sigma 
Aldrich G5516-500ml) with food grade NaCl until 
Saturation. The solution was then carefully poured into a 
syringe through a filter to prevent any undisolved NaCl 
crystals from being captured. For ease of 

experimentation, the sensor channel geometry was 
manufactured using high purity silicone microtubes with 
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a 305pm inner diameter, and 150pm wall thickness 
(McMaster-Carr #51845K65). These tubes were cast 
into a 1.1mm thick layer of PDMS, then injected with 
NaCl saturated glycerol with a 30g blunt tip needle. 
Electrodes composed of 280pm diameter nickel- 
titanium wire were used for corrosion resistance and 
biocompatibility. These electrodes were inserted 
through the glycerol filled micro-tubes, then tied off 
with 5-0 braided silk sutures to prevent leaks. Finally, 
the sensor was rinsed with isopropanol, then cast in 
another layer of PDMS to fully seal the liquid channels 
to the electrode junction. Fig 2 shows a semicircular 
sensor design, with a radius of curvature of 1mm, for 
highly localized pressure sensing. The oscillation 
circuit as proposed by [5] was used to generate an AC 
square wave. The conductivity of NaCl saturated 
glycerol is approximately 2 Orders of magnitude lower 
than that of saturated saline, which necessitated a rail- 
to-rail voltage of +/-2.5V for stable Operation. 



Fig. 2 Sensor prototype. The sensor is 1.1mm thick and uses 
glycerol dyed with food colouring for channel visualization. 

Two experiments were performed using the sensor 
prototype shown in Fig. 2. The first experiment 
measured the change in oscillator period as a function of 
applied force using a 2.5mm diameter PDMS indenter. 
The second experiment evaluated the longer-term 
stability of the sensor in terms of robustness to 
evaporation, electrolysis and electrode corrosion. For 
this test, the sensor was put into continuous use for 14 
hours with measurements of base resistance and sensor 
function made approximately every hour to ensure that 
no adverse electrochemical effects were present. 

RESULTS 

The applied force versus measured oscillator period is 
plotted in Fig. 3. The depicted curve is a two-term 
exponential function F = 18.3exp(0.59T) - 2785exp(- 
12.4T), where F is the load in grams, and T is the period 
in milliseconds. Channel collapse was observed at ~43g 
load, yielding a loss of oscillator signal. Though the 
data was taken at increments of approximately 5 g, it 
was possible to detect loads as low as ~2g based on the 
resolution of the oscilloscope’s frequency analyzer. 


In the second test, no bubble formation due to 
evaporation or change in base resistance was noted over 
the 14 hour test period. 



Fig. 3 Applied Force vs. Oscillator Period 

DISCUSSION 

NaCl saturated glycerol is a suitable biocompatible 
Substitute for eGaln in soft sensors for robotic surgical 
applications. The experiments discussed herein 

demonstrate that it is possible to fabricate a soft sensor 
using biocompatible materials exclusively. Future work 
will include: 1) sensitivity comparison between 

biocompatible ionic-based sensors and their eGaln- 
based predecessors 2) optimization and refmement of 
the ionic solution to maximize conductivity and 
sensitivity 3) development of prototype Support 
electronics with a focus on a low production cost, and 4) 
Integration of the sensing System with a surgical robot 
platform, for bench-top and in- vivo testing. 

Towards clinical adoption, we envision the use of 
biocompatible soft sensors with inexpensive electronics 
for amplification and sampling to be a disposable add- 
on to existing surgical robotic Systems. 
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INTRODUCTION 

Foreign bodies in the heart (e.g. thrombi, shrapnel) can 
cause a series of adverse Symptoms, such as arrhythmia, 
cardiac tamponade, hemorrhage, infection, embolism, 
valve dysfunction, and anxiety, and have to be removed 
surgically ’ . The most frequent surgical approach for 
removing foreign bodies from the heart is a median 
sternotomy followed by an incision in the heart muscle 
to gain direct access to the object ” . Median sternotomy 
is a highly invasive procedure imposing risks such as 
bacterial mediastinitis, inflammation, bone fracture, and 
vulnerability to future ehest injury. Cardiopulmonary 
bypass (CPB), which is often used to stop the heart 
during surgery, is associated with risks of hemolysis, 
clotting, and air embolism, and can lead to extended 
durations of procedures as well. 

To address the disadvantages of the open surgery, 
we propose a minimally invasive, transapical approach 
for retrieving foreign bodies from a beating heart. The 
task is carried out using a dexterous robotic capture 
device, with 3D transesophageal echocardiography 
(TEE) providing internal visualization of the otherwise 
occluded heart chamber, foreign body, and robot. Under 
intraoperative ultrasound guidance, the robot moves to 
secure the target; the setup is illustrated in Fig. 1. This 
proposed approach represents the initial steps in 
improving the state of care by reducing risk and 
mortality, improving postoperative recovery, and 
potentially reducing operating room times. 

In our previous work, we tracked a foreign body in a 
beating heart phantom to study its behavior . We then 
found that speed of the foreign body is too fast for a 
robot to directly pursue it 8 . To provide a relatively slow 
robot with the ability to retrieve an erratically moving 
target, we proposed the concept of guiding a robot to a 
selected location and ambushing the target upon its 
reappearance to that location. We examined alternate 
indirect capture strategies ’ that relax the level of 
performance required of a robot. Leading into real-time 
capture experiments we studied the time evolution of 
computed capture locations 10 . 

In this paper, we demonstrate the use of a high 
dexterity, minimally invasive surgical robot to retrieve a 
foreign body from a beating heart phantom. Streaming 
3D TEE imaging is used to track the foreign body, 
compute a suitable capture location, and guide the robot 
to secure it. This effort differs from related works that 
focus on rigid tools or on manual guidance. 

This work was funded in part by Philips Research North America 
and in part by Johns Hopkins University internal funds. 


Dexterous Manipulator 



Fig. 1 Minimally invasive robotic retrieval of foreign bodies 
from the heart under 3D transesophageal echocardiography 
(TEE) guidance. 


MATERIALS AND METHODS 

The experimental setup (Fig. 2) consists of a Philips 
iE33 xMATRIX Echocardiography System and X7-2t 
3D TEE probe, a beating heart phantom, and a high 
dexterity robot. The robot consists of a seven-degree of 
freedom (DOF) IBM/JHU Lars robot holding a JHU 
snake robot, a 39.5-mm long, 4.2-mm diameter 
dexterous end effector with four distal DOF. The robot 
is inserted into the left ventricle of the heart via the 
apex. Fig. 3 shows this arrangement in detail. 

A 3.2-mm Steel ball representing a foreign body such 
as shrapnel is placed in the phantom, and the scene is 
imaged using the ultrasound probe at a rate of 20 
volumes per second. Tracking of the foreign body is 
performed using a modified normalized cross- 

n 

correlation method; previous reports describe this 
process in greater detail. 
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Fig. 2 Equipment setup for real-time capture experiments. 
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and 46.3 seconds. We reported an observation time for 
this method of roughly 20.0 seconds previously 10 , while 
noting large variations between trials. Such variability 
can be explained by the irregulär motions traced by the 
foreign body. In light of these considerations, the values 
obtained in this study appear to be within reason. 

The retrieval process was completed in 97.7 ± 21.6 
seconds. Though governed largely by human discretion, 
this figure serves as a reference in the context of cardiac 
procedures, which often last several hours. Potentially, 
the perioperative time incurred by robotic aspects would 
be minimal, and more sophisticated techniques can be 
used to improve outcomes. These results demonstrate 
the viability of using a slow, dexterous robot to capture 
an erratic cardiac target under 3D ultrasound guidance. 
Future work will include workflow optimization, 
sterility, tooling for capturing multiple types of targets, 
and possible Integration of real-time X-ray imaging. 

REFERENCES 

[1] Marshall AJ, Ring NJ, Newman PL. An unexplained 
foreign body in the myocardium. J. Royal Soc. Med. 
2002; 95:250-251. 

[2] Symbas PN, Picone AL, Hatcher CR, Vlasis-Hale SE. 
Cardiac missiles. A review of the literature and personal 
experience. Ann. Surg. 1990; 211(5): 639^18. 

[3] United States Department of Defense. Emergency War 
Surgery: Third US Rev, s.l., 16: Thoracic Injuries 2004. 

[4] Actis Dato GM, Arslanian A, Marzio PD, Filosso PL, 
Ruffmi E. Posttraumatic and iatrogenic foreign bodies in 
the heart: report of 14 cases and review of the literature. 
J. Thorac. Cardiovasc. Surg. 2003; 126(2): 408^114. 

[5] Evans J, Gray LA, Rayner A, Fulton RL. Principles for 
the management of penetrating cardiac wounds. Ann. 
Surg. 1979; 189(6): 777-784. 

[6] Nessen SC, Lounsbury DE. War Surgery in Afghanistan 
and Iraq: A Series of Cases, 2003-2007, s.l. Department 
of the Army, Office of the Surgeon General and Borden 
Institute, 14: Thoracic Trauma 2008. 

[7] Thienphrapa P, Elhawary H, Ramachandran B, Stanton 
D, Popovic A. Tracking and characterization of 
fragments in a beating heart using 3D ultrasound for 
interventional guidance. MICCAI 2011; 6891: 21 1-218. 

[8] Thienphrapa P, et al. A System for 3D ultrasound-guided 
robotic retrieval of foreign bodies from a beating heart. 
IEEE/EMBS BioRob 2012; 743-748. 

[9] Thienphrapa P, Ramachandran B, Elhawary H, Taylor 
RH, Popovic A. Multiple capture locations for 3D 
ultrasound-guided robotic retrieval of moving bodies 
from a beating heart. SPIE Medical Imaging 2012; 8316. 

[10] Thienphrapa P, et al. Intraoperative analysis of locations 
for 3D ultrasound-guided capture of foreign bodies from 
a beating heart. Hamlyn Symp. Medical Robotics 2012. 


(a) (b) (c) (d) (e) 



Fig. 4 Image sequence showing different phases of foreign body capture. (a) Foreign body (circle) in the heart ventricle just before a 
capture location is determined. (b) Dexterous robot (arrow) approaches capture location. (c) Robot captures foreign body. (d) Robot 
leaving heart with foreign body attached. (e) Heart empty after foreign body is extracted. 



Fig. 3 Detailed view of the experimental setup showing the 
Position of the TEE probe, beating heart phantom, and snake 
robot inserted into the heart at a point near the apex. 


EXPERIMENTS 

The attractivity of a small 1.6 x 3.2 mm magnet was 
tested by advancing the magnet towards the foreign 
body and measuring the distance at which attraction 
occurred; this ränge was found to be 4.2 ± 0.8 mm over 
20 repetitions. The magnet is affixed to the tip of the 
snake robot to represent an abstract capture device with 
high attractivity within the capture ränge and negligible 
effect beyond it. A one-time preoperative registration is 
done to relate the coordinate Systems of the ultrasound 
volumes and the robot. 

Commencing each trial, the robot is teleoperated in 
coarse control mode to place the tip of the snake robot 
at the entry point of the beating heart phantom. This 
establishes the virtual remote center of motion (RCM), 
from which point the robot operates autonomously. 

Then, the foreign body is interactively selected from 
still images to define a template to be tracked in the live 
image stream. Computation of capture locations based 
on spatial probability is then initiated. Once a suitable 
capture location is found, the tracker commands the 
robot to said location. When the end effector reaches its 
destination, visual inspection is used to determine 
whether the foreign body has been captured. The robot 
is then withdrawn from the beating heart phantom, and 
success or failure of the attempt is recorded. 

RESULTS AND DISCUSSION 

An example foreign body capture sequence is shown in 
Fig. 4. The foreign body was extracted successfully in 
14 out of 17 attempts (82.4%). In two of the three failed 
attempts, the foreign body shifted to a different location 
after a capture location was found; this phenomenon has 
been noted in our prior studies. The other failed attempt 
can be attributed to premature withdrawal of the robot 
before it completed its capture trajectory. 

The mean observation time before executing a 
capture was 29.6 ± 6.9 seconds, with extrema of 21.6 
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INTRODUCTION 

Virtual autopsy, or Virtopsy® [1], originated from the 
goal of implementing diagnostic techniques such as 
computed tomography (CT), magnetic resonance 
imaging (MRI), three-dimensional (3D) photo- 
grammetry and surface scanning, and post mortem 
minimally invasive biopsy, in order to objectify medico- 
legal investigations [2]. The Virtopsy® concept was 
presented in 2000 as a systematic approach to 
complement or substitute Standard forensic procedures 
[3]. For further optimization of the developed work- 
flow, a first prototype of a robotic System for post- 
mortem surface scanning and image guided biopsy has 
been developed and implemented at the Institute of 
Forensic Medicine in Bern, Switzerland, in 2008 [4]. 
Based on the experience from this System setup, an 
enhanced prototype was developed at the Institute of 
Forensic Medicine, University Zürich, Switzerland, and 
Operation started in July 2012. This paper describes the 
Virtobot robot System in general and the new System 
functionality in particular. In addition, the paper reports 
some first performance data and experiences from 
routine use. 

MATERIALS AND METHODS 

The Virtobot System serves as a multifunctional robotic 
System capable of performing major tasks of the 
Virtopsy® work-flow. A six axis robotic arm (Stäubli 
TX90L, Stäubli International AG, Pfäffikon/CH) is 
combined with a tailored ceiling-mounted linear guide 
unit (ränge of motion: 4 meters) which is located above 
the CT gantry and aligned with the CT patient couch. 
The base of the robotic arm is mounted onto a dedicated 
lifting axis to allow moving the robotic arm to a parking 
Position above the CT-gantry in order to deal with the 
specific space restrictions. Different from the first 
prototype System in Bern, a fully automated tool 
changing unit now allows the robotic manipulator to use 
corresponding end-effectors to perform different tasks 
of the Virtopsy® work-flow. Figure 1 shows the three 
different end-effectors that are currently included and 
stored in a tool magazine Station. In the present 
configuration, the following three tools are available: a 
precise optical 3D surface Scanner (GOM ATOS 
Compact Scan, Gesellschaft für optische Meßtechnik 
GOM GmbH, Braunschweig/D), a digital SLR camera 


(Nikon D700, Nikon Corporation, Tokyo/JP), as well as 
a custom made biopsy tool (Integrated Microsystems 
Austria GmbH, Wiener Neustadt/ A). Figure 2 shows the 
robotic arm equipped with the 3 D- surface Scanner that 
generates a high resolution model of the corpse in order 
to document pattemed injuries. Scanning takes place in 
5 scanning lines with 5 points each. Both lines and 
points can be specifically adapted via the control 
Software. The surface data can be enhanced by adding 
texture data captured with the SLR camera. Aside from 
documentation purposes, 3D surface data can be used 
for virtual reconstructions of the incident [5]. 



Fig. 1 Virtobot tool magazine. From left to right: 3D surface 
Scanner, digital SLR camera, biopsy tool. 



Fig. 2 Virtobot arm equipped with 3D surface Scanner. 


Figure 3 shows the custom made biopsy tool that allows 
the automatic and precise placement of coaxial 
introducer needles based on CT data. The needles can 
be used to harvest tissue or liquid samples and place 
guide wires to assist pathologists in finding small 
foreign bodies during autopsy. 

A set of Software packages is integrated into the 
Virtobot setup for different tasks. The Virtopsy® 
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control center (VCC) Coordinates the entire work-flow 
and Controls the robot. Düring surface scanning, pre- 
programmed scan positions and photogrammetric photo 
positions can be selected in the VCC, and are processed 
automatically. In addition, the robotic arm can be 
manually controlled in joint or tool Coordinates to 
provide additional grade of flexibility in choosing 
special scan/photo positions. The performed Virtopsy® 
work-flow is been logged and all captured data are 
stored. The GOM application Software V7.5 SRI 
operates the 3D surface Scanner and the photo- 
grammetry camera and processes the data. It also 
calculates the fully textured true to scale 3D polygon 
model from the surface of the skin. 



Fig. 3 Biopsy tool attached to the Virtobot arm. 



wounds, one knife wound, and one case of 
Strangulation. Six of these cases are included in court 
forensic reports. One particular advantage of the use of 
the Virtobot System is the significant increase of 
efficiency [4]. A manual surface scan of one side of a 
victim using a Standard tripod takes approximately 30 
min. By using Virtobot, this time could be reduced to 
approximately 15 min. 

For evaluation of the accuracy of needle placement, 
a preliminary accuracy study was conducted based on a 
biopsy phantom [4]. Calibration of the robot was 
performed with six pivot points and a mean RMS of 2.3 
mm (±0.04 mm, n = 13). The cone centre for every 
pivot point was determined with an accuracy of 0.5 mm 
(± 0.06 mm, n = 78). Two phantoms were prepared and 
a total of 13 needles set. The mean accuracy was 3.2 
mm (±1.9 mm, n - 13). Up to now automatic needle 
placement has been performed on three real cases to 
obtain body liquids like urine or blood and retrieve 
tissue samples from organs of interest. 

DISCUSSION 

The Virtobot System automates tasks that are either 
repetitive or require high accuracy. It allows fully 
automated surface documentation and post-mortem 
minimally invasive image guided needle biopsy. Two 
prototypes of the robot System have been realized so far. 
The demonstrated reduction of process time for the 
Virtopsy® work-flow by implementation of Virtobot 
could be further improved by updating the navigation 
Software with an automatic registration and automation 
of the tool change. Within the Virtopsy® project, the 
Virtobot System is a valuable tool to gather objective 
data in a minimally invasive manner. While surface 
documentation of pattemed injuries allows for incident 
reconstructions, image guided biopsy contributes to 
correlate radiological with histological fmdings, which 
would be impossible if tissue samples are taken during 
autopsy. In the future it might be possible to use the 
Virtopsy® fmdings to determine if a further autopsy is 
necessary to gain additional evidence in forensic cases. 


Fig. 4 Navigation Software for tissue sample path planning. 

The surgical navigation Software allows for biopsy 
trajectory planning. Passive optical markers and an 
infrared tracker are used to determine the position of the 
biopsy tool and the body in space. For registration of the 
CT dataset with the actual body, radiopaque tungsten 
balls are placed at the marker center and automatically 
detected by the navigation Software. During trajectory 
planning, target and entry point, as well as the 
orientation of the end-effector are defmed. (Figure 4) 

RESULTS 

Both Virtobot Systems are used in daily routine. Up to 
now the Bern System has been used in 52 cases; the 
System in Zürich has provided robotic assistance for 
automated surface scanning in 16 cases: six victims of 
traffic accidents, two cases of blunt trauma, six gunshot 
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ABSTRACT 

We describe the early clinical application of a robotic 
transrectal ultrasound (TRUS) System for real-time 
monitoring of the prostate and periprostatic anatomy 
during robot-assisted laparoscopic radical prostatectomy 
(RALRP). Two patients with clinically organ-confmed 
prostate cancer undergoing RALRP at our Institution 
agreed to participate in this CREB-approved (clinical 
research ethics board) pilot study. Before docking the da 
Vinci robot, our robotic TRUS System [1] was attached 
to the foot of the operating table using the MicroTouch 
Brachytherapy stabilizer passive arm (CIVCO Medical 
Sloutions, Kalona, IA). A bi-plane TRUS transducer 
(Ultrasonix Medical Corporation, Richmond, BC) was 
placed and adjusted to provide optimal trans verse and 
sagittal images of the prostate and its surrounding 
structures. The set-up of the TRUS robot and the da 
Vinci in the operating room is shown in Fig. 1. 

The TRUS robot was used during the procedure to 
remotely control the rotation angle and insertion depth 
of the TRUS transducer to provide a full view of the 
prostate even when the anatomy was shifted during the 
procedure. Furthermore, a calibration technique [2] was 
performed between the TRUS transducer and the da 
Vinci instrument, to enable the TRUS imaging plane to 
automatically track the tip of the da Vinci instrument. In 
order to achieve automatic tool tracking, the surgeon 
was asked to press the tool tip of a da Vinci instrument 
(EndoWrist Monopolar curved Scissors) against the 
anterior surface of the prostate in four locations. The 
instrument tip, visible as a hyperechoic focal point in 
the B-mode images, was then located in the TRUS 
volume and the homogenous transformation relating the 
da Vinci coordinate System to that of the TRUS was 
obtained as previously explained in [2]. The calibration 
procedure was completed in less than two minutes and 
was valid throughout the surgery with an accuracy of 
within 2 mm as computed in our previous in-vivo 
animal study [3]. With such a tracking accuracy, 
movement of the tool tips was constantly visible in the 


ultrasound image, showing the location of the tools 
relative to the target anatomy (Fig. 2 d-f). During each 
procedure, real-time TRUS images were relayed to the 
surgeon and used to precisly identify the location of the 
bladder neck, the apex, and the dissection planes of the 
neuro-vascular bundles (NVB). Automatic tool tracking 
was activated during all critical stages of the surgery to 
faciliate real-time TRUS navigation using the same 
instruments being used during the procedure. 

Complete robotic TRUS evaluation was successful 
in both patients. No major complications occurred and 
the transducer remained in place until the end of the 
procedures. We conclude that real-time robotic TRUS 
guidance during RAFRP is feasible and safe and it can 
provide the console surgeon with valuable anatomic 
information, with the aim of maximizing oncologic and 
functional outcomes. The automatic tool tracking 
System provided an easy to use TRUS guidance System 
to the surgeons since the surgical tool was used to 
automatically control and adjust the imaging plane, 
making it an easy and intuitive System for surgeons to 
use. 
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Fig. 1 Intraoperative robotic TRUS during RALRP, (a) TRUS robot attached to the operating table in Trendelenburg position before 
docking the da Vinci System, (b) TRUS transducer and robot attached to the foot of the operating table using the MicroTouch 
Brachytherapy stabilizer passive arm, (c) operating room setup of the TRUS robot and the da Vinci System. 



Fig. 2 Intraoperative TRUS images of the prostate and its surrounding structures, (a) prostate and seminal vesicles in sagittal B-mode 
TRUS image, (b) transverse B-mode image of the prostate gland, (c, d) prostate and NVB visualization using Bmode and Doppler 
ultrasound, (e, f, g) da Vinci instrument tip visible in the TRUS images in different stages of the procedure while the automatic tool 
tracking was activated, (e) da Vinci instrument tip in transverse US image. 
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INTRODU CTION 

Wire drive mechanism is widely used for forceps with 
multi degrees of freedom. However the backlash and 
stick slip phenomena found in the mechanism made it 
difficult to control the makanism precisely. Kaneko et al 
proposed physical model to represent effect of friction 
and elongation of the wire in tendon driven mechanism 
with flexible sheath 1 . It was demonstrated that the 
amount of backlash is dependent on wire tension. The 
Stick-slips motion sue to static friction of the joint also 
affects the amount of backlash. Since Stick-slip motion 
is inherently unstable, it is difficult to estimate the 
amount of backlash under various loading conditions. In 
this study, we proposed a new method to compensate 
backlash under various loading conditions. 

MATERIALS AND METHODS 
Experimental System 

To investigate changes in wire displacement and tension 
of the wire, the experimental System shown in Fig.l was 
constructed. In the present study, pre-tension was 
provided by an electric motor where the generated 
torque by the electric motor was control to be constant 
by PID Controller. Wire displacement was estimated by 
the rotation angles of the two electric motors for 
actuation side and pre tension generation side. The 
tensions of the both wires were measured by two load 
cells installed in both sides as shown in Figure 1. One 
motor was used to control the wire tension, and the 
other motor was used to set the rotation angle of the 
gear. The forceps mechanism used is shown in Figure 2. 
Two load cells can be used also for the estimation of 
applied force on two blades. 

We used forceps mechanism proposed by Nishizawa 
et al. [2] as shown in Fig. 2. In this mechanism, the path 
length of the wires for the two tip blade is kept constant 
when the bending is generated by the mechanism. 
Bending of the mechanism is realized by rotating the 
gear 1 in Fig. 3 using two wires attached to it. 

Backlash compensation 

Generated torque by the electric motor was adjusted 
constant by a PID Controller. When the direction of 
the gear’s rotation is reversed, the motion of the gear 
does not follow the displacement of the wire since static 
friction of the joint prevents the gear from moving. The 
initiation of the reverse movement can be identified by 
monitoring the displacement of the wire of the tension 
control side. When the gear Starts to move in reverse 


direction, abrupt change in wire displacement of the 
wire is observed in the wire tension control System. In 
this study, we utilized this phenomenon to detect the 
initiation of reverse motion. The amount of backlash 
due to this stick slip motion was estimated as the 
amount of displacement of wire of the position control 
side before the initiation of movement of the wire of the 
torque control side. 




AD Converter 


Motion Controller 


DA Converter 
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Motor 


i 


Computer 



Fig. 1 Experimental System 
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Fig. 2 Mechanism of the 3 DOF forceps 


RESULTS 

Fig.4 shows the changes in displacement of wire of the 
Position control side expressed as “input angle”, the 
bending angle of the forceps, and the rotation of the 
motor Controlling the wire tension expressed as “Motor 
Speed”. The bending angle was estimated by measuring 
distance between the plate attached at the tip of the 
forceps and laser displacement meter (LB-62 A 
KEYENCE Tokyo, Japan). Wire tension was controlled 
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as large as ION. The abrupt change in motor speed of 
the tension control motor indicates the initiation of the 
joint reverse motion. 


Tip part 



Root part 


Fig.3 mechanism of bending joint to keep the path length of 
wires for the tip blade actuation [2] 


mechanism to detect stick-slip motion should be 
adopted in practical design. On the other hand, 
reduction of stick-slip motion by adjusting the wire 
tension by two electric motors and estimation of 
external load can be implemented based on quantitative 
measurement of both wires’ tension. 
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Fig. 4 Change in displacement of wire of the position control 
side expressed as “input angle”, the bending angle of the 
forceps, and the rotation of the motor that Controls the wire 
tension expressed as “Motor Speed” 

Fig. 5 shows the backlash compensation without any 
load at the tip of the forceps. The joint was actuated to 
represent triangulär waveform to generate forward and 
backward motion. Without compensation, there was 
significant hy stere sis between the input angle and 
resultant output angle. 

Fig. 6 shows the backlash compensation with load as 
large as 50gf at the tip of the forceps. The joint was also 
actuated to represent triangulär waveform to generate 
forward and backward motion. 

DISCUSSION 

We proposed a method to compensate backlash based 
on the detection of disturbance in wire tension control 
System. The amount of backlash due to static friction 
leading to stick-slip motion was estimated as the amount 
of wire displacement before the initiation of motion in 
reverse direction. We used relatively complicated 
measurement System using two load cells and two 
motors in the present experimental System. Simpler 


Fig. 5 Input angles and resultant bending angles of the joint 
without any load at the tip of the forceps (wire tension: ION) 



Fig. 6 Input angles and resultant bending angles of the joint 
with load as large as 50 gf at the tip of the forceps (wire 
tension: ION) 


REFERENCES 

[1] M Kaneko, M. Wada, H. Maekawa, and K. Tanie: 
Considerations on Force Servo System of Tendon Drive 
Robotic Hands - Transmission Characteristics of Driving 
System and Their Influences, Journal of Robotic Society 
of Japan, vol.9(4) 437-444, 1991 

[2] K. Nishizawa, K. Kishi, F. Tajima, K. Sudo, M. Fujie, S. 
Takamoto, and T.Dohi: Principles and Applications of 
Interference -Free Wire Driven Joint Mechanism, JSME 
annual meeting 2004(5), 97-98, 2004. 


The Hamlyn Symposium on Medical Robotics (2013) 


14 




A Dexterous Instrument for Minimally Invasive Neurosurgery 

F. Khan, B. Carrillo, T. Looi, J. Drake 

Center for Image Guided Intervention and Therapeutic Innovation 

The Hospital for Sick Children, Toronto, Canada 

fouzia. khan@sickkids. ca 


INTRODUCTION 

The state of the art instruments for minimally invasive 
neurosurgery, discussed in [1], only have 2 degrees of 
freedom (roll, and translation) which limits the dexterity 
and the operating workspace. This reduces the variety of 
procedures that can be performed in a minimally 
invasive manner and increases the difficulty in avoiding 
critical structures. Enhancing minimally invasive 
neurosurgery with a higher degree of freedom 
instrument is the motivation for developing a 4 degree 
of freedom robotic instrument. This paper presents the 
kinematics and the preliminary experimental results of 
that instrument. 


to cartesian co-ordinates (x, y, z, Orientation), is given in 
equations [1] and equations [2]. 

orientation = yaw (1) 

where, orientation is the angle between the z- axis of the 
end disk co-ordinates and the z-axis of the gripper co- 
ordinates measured about the y-axis of the end disk co- 
ordinates. The gripper co-ordinates are overlapping with 
that of the end disk co-ordinates in Figure [1]. 


-x- 


0' 

y 

-z- 

= R z (roll)p(bend) + 

0 

-t. 


SYSTEM OVERVIEW 

The System consists of 
a master device, an 
actuation unit and a 
flexible instrument. 
The master device 
provides 3D position 
and ID orientation 
that is the reference 
for the instrument’ s 
tip. The actuation unit 
consists of motors to 
drive the flexible 
instrument that is 
based on the design of 
the distal units 
presented in [2] and 
[3]. A schematic of 
the instrument is 
shown in Figure 1. 
The primary component of the instrument is the super 
elastic nitinol tube called the primary backbone. This 
backbone can be bent by applying tension on the three 
wires that are routed using spacer disks along the 
backbone. The bend is defined as the angle between the 
z-axis of the end disk co-ordinates and the x -axis of the 
base disk co-ordinates. It can be translated and rotated 
about the z-axis of the reference co-ordinates. A gripper 
with 1 degree of freedom is affixed to its end. Thus, the 
instrument has 4 degrees of freedom; bend, roll, 
translation, and a yaw that is at the gripper. 

FORWARD KINEMATICS 

The forward kinematics, the mapping from the 
instrument’ s parameters (bend, roll, translation and yaw) 



flexible instrument 


where, 


R z (roll) = 


cos (roll) —sin (roll) 0 

sin (roll) cos (roll) 0 

0 0 1 


p (bend) = -w 


— - bend \ 


mag(bend)(l — sin (bend) 

0 

. mag(bend)(cos(bend)) 


t = translation 


mag(bend) 


(f-bend) 

|^-bend| 


/ = length of the instrument 

Equations [1], and [2] have been derived under the 
assumption that the shape of the instrument has a 
constant curvature. In addition, the parameter bend is 
assumed to be between 0 and n. 

INVERSE KINEMATICS 

The inverse kinematics, the mapping from Cartesian co- 
ordinates (x, y, z, orientation) to the instrumenfs 
Parameters (bend, roll, translation and yaw), is given in 
equations [3] - [6] 

yaw = orientation (3) 

roll = tan -1 (4) 

bend = min ö ( k(b) — -Jx 2 + y 2 ) (5) 
translation = z — zb (6) 

where, 
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k(b') = 


l 


n u 
2~ b 


(mag(b)( 1 — sin(fe))) 


mag(b ) 



zb 



(d - {x 2 + y 2 )) 



(1 — sin (bend)) 


l = length of the instrument 


CONTROL 

A feedforward control is implemented on the prototype. 
Figure [2] shows a block diagram with the important 
functions of the control block. The inverse kinematics 
shown in the previous section is utilized to determine 
the instrument’ s parameters. The parameter bend needs 
to be adjusted as there is a discrepancy between the 
constant curvature assumption in the model and the 
curvature of the physical instrument. The adjustment 
function f(bend) is found experimentally and it is a 
linear function. Given the parameters, the displacement 
of the motors (< q 1} q 2 , ^ 3 ) that actuate the instrument can 
be found using equations [7] and [ 8 ]. Displacement of 
the other motors ( q 4 , q 5 , q 6 ) are linear functions 
determined based on motor parameters. 


q 1 = r [bend — 

1 

= Vs = ~2 r 




where, r is the distance from the center of the spacer 
disk to one of the wires. 



Fig.2 Control Block Diagram 


parameter with a maximum error of a quarter degree. 
The bend of the instrument was measured visually using 
a protractor. This test was conducted 10 times for the 
full ränge that the instrument can bend. A preliminary 
Position accuracy test was conducted that focused on 
Position error of the instrumenfs tip on the x-y plane. 
The position error on the z axis was not tested because it 
is the same as the position error of the linear motor that 
Controls the z-axis motion. The reference position 
ranged from 6 mm to - 6 mm with a resolution of 1 mm on 
both x and y axis. The instrumenfs tip position was 
measured using the NDI Aurora Electromagnetic 
tracker. The Euclidean norm of the vector difference 
between the reference and the measured position was 
used as the position error. The test was repeated twice 
and the average error was 4mm. Possible sources of 
position error are the discrepancies between the model 
and the physical instrument, or metallic interference 
from the actuation unit in the electromagnetic field. 
Feedback control would mitigate the model discrepancy 
issue, when the discrepancy is small. In order to resolve 
the metallic interference, plastic material can be used 
for the body of the actuation unit or a different tracking 
technology like infrared or charged coupled device can 
be utilized. Other methods to improve the accuracy will 
be investigated in the future. 



Fig.3 Instrument prototype 


CONCLUSION 

This paper presented a four degree of freedom robotic 
instrument for neurosurgery. A prototype of the 
instrument, with dimensions suitable for pediatric 
neurosurgery, has been built. It can be controlled using 
a master device. Preliminary accuracy evaluation shows 
the mean error between the instrumenfs tip and the 
reference position to be 4 mm. Future works will 
involve improving the position accuracy, increasing the 
degrees of freedom and path planning for the flexible 
instrument. 


EXPERIMENTAL RESULTS 

The prototype of the instrument, shown in Figure [3], is 
2.2 mm in diameter and 28 mm in length. The master 
device is the Novint Falcon; the Controller is 
implemented in MATLAB running on an Intel i7 3.4 
GHz processor with a Windows 7 operating System; 
communication between the Controller and the actuation 
unit is through NI-PCI6224 card. The instrumenfs tip 
tracks the position of the Falcon’s handle and the 
Orientation of the tip can be controlled using the buttons 
on the Falcon’s handle. A demonstration of the 
instrument can be found at http://youtu.be/OidxryI3-Rc. 
The instrument can track reference signals for its bend 
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INTRODUCTION 

Cochlear implantation, for the treatment of moderate to 
severe hearing loss, requires access to the inner ear for 
electrode insertion into the cochlea. Access is gained via 
a mastoidectomy in which large amounts of tissue are 
removed to enable visualisation and thus, preservation 
of close lying critical anatomy such as the facial nerve. 
To replace the mastoidectomy, a minimally invasive 
approach, in which access is alternatively gained 
through a tunnel approximately 20 times smaller (1.8 
mm in diameter) in a procedure termed direct cochlear 
access (DCA) [1], has been proposed. DCA has the 
potential to significantly reduce invasiveness and 
improve surgical outcomes for hearing device 
implantation procedures. By replacing the surgeon’s 
vision with a surgical plan based on preoperative 3D 
imaging, an image guided robotic procedure could 
potentially facilitate this previously unattainable 
minimally invasive procedure. Realisation is, however, 
reliant on the ability to reduce Standard image guided 
positioning errors tenfold (less than 0.3 mm). 

Whilst image guidance and surgical robots such as 
the DaVinci have aided in the conduction of 
conventional surgical techniques; improving surgeon 
confidence and precision for decades, to date, neither 
technology has been successfully applied to 
microsurgical applications, primarily due to an inability 
to obtain the required accuracy and safety levels. 
Additionally, alternative techniques for intraoperative 
risk assessment throughout the conduction of the 
procedure are required to cope for a loss of direct visual 
or video feedback. Herein, we describe the concept, 
design and validation of an image guided robotic System 
that would enable minimally invasive cochlear 
implantation through the facilitation of the DCA 
procedure. 

MATERIALS AND METHODS 

An image guided robotic System dedicated to the 
drilling of a DCA tunnel has been developed (Fig.l). 
The lightweight (5.5 kg), five degrees of freedom serial 
robotic arm was designed to attach directly to the 
operating table and conducts the surgical procedure 
based on a preoperative plan. In order to achieve the 


required high positioning accuracies, techniques that 
enable error reduction in each of the three primary 
phases of the procedure (planning, registration and 
drilling) have been developed. Additionally, procedural 
safety information is calculated and fed back to the 
surgeon for assessment during each phase. 

The robotic System is designed to drill a minimally 
invasive tunnel from the mastoid surface to the cochlea 
using the following workflow. Initially, four 3 mm 
surgical screws (M-5220.03, Medartis, Switzerland) are 
implanted in the mastoid as registration fiducials. 



| No n-invasivehead holder 


Fig. 1 Image guided robotic System. 

Secondly, a preoperative plan is produced from high 
resolution cone beam CT (CBCT) images with the aid 
of a dedicated Software tool [2]. From the CBCT image 
dataset, critical anatomical structures to be preserved 
(facial nerve, chorda tympani, ossicles, ear canal wall) 
along with the mastoid bone are segmented. A trajectory 
is then defined from the mastoid surface to the cochlea, 
avoiding the surrounding critical anatomy (Fig. 2). 
Distances from the defined tunnel to segmented 
anatomical structures (accounting for predicted case 
specific registration error and average tracking error) 
are calculated and displayed to the user to aid in risk 
and patient eligibility assessment. The fiducial screws 
are localized within the image using a semiautomatic 
detection algorithm, described in [3]. The surgical plan 
is thereafter transferred to the robotic System. 


The Hamlyn Symposium on Medical Robotics (2013) 


17 



Intraoperatively, the robot is mounted to the 
operating table side rail and the patient’ s head is 
immobilized. The robotic end effector is tracked relative 
to the patient in 3D space with a mean accuracy of 0.05 
mm by a high accuracy optical tracking camera 
(CamBar Bl, Axios, Germany) via custom made active 
optical markers that are attached to the tool and 
anchored to the mastoid. 

The patient is registered to the preoperative plan via 
detection of the implanted fiducial screws using the 
robotic arm, the optical tracking System and an 
automatic tactile search algorithm, as per the 
methodology described in [3]. The registration process 
provides registration errors at the target of 
approximately 0.1 mm [3]. The preoperatively planned 
trajectory is thereafter drilled by the robotic System 
using the optical tracking camera for position control. 

Throughout the drilling process the pose of the tool 
is verified using a force-density correlation algorithm 
based on real time drilling force data and image density 
profiles obtained from preoperative CT around the 
drilling site [4]. Tool position verification is displayed 
to the user along with the optically tracked drill position 
on the graphical user interface (GUI) within a virtual 3D 
view of the surgical scene and within a CBCT slice 
viewer. The GUI, which is also used for navigating the 
surgical workflow, is displayed on a touch screen 
monitor positioned beside the operating table. 

A manual incision is made in the round window with 
a small knife, and the electrode is manually inserted 
through the DCA tunnel into the cochlea. 


Fig. 2 Postoperative accuracy evaluation of a robotically 
performed DCA on a cadaver head. 


The image guided robotic System and the proposed 
surgical approach were evaluated in eight cases 
performed on four clamped (Mayfield, USA) whole 
cadaver heads. Two further cases were rejected during 
the preoperative planning phase due to an excessively 
narrow space between the facial nerve and the external 
auditory canal wall. In each case, the round window was 
targeted and a Standard 22 contact electrode (Med-El, 
Austria) was manually inserted through the DCA tunnel 
and into the cochlea. Postoperative CBCT data was used 
to analyse the accuracy of the drilled tunnel and the 
effectiveness of the electrode insertion. 


RESULTS 


DCA tunnels were drilled to the middle ear cavity with 
mean accuracies at the mastoid surface and target of 
0.08 ± 0.05 mm and 0.15 ±0.08 mm (N = 8) 



respectively. 

Visual inspection of postoperative images showed 
that all critical structures were preserved in all cases 
(Fig. 2). Electrodes were inserted through the round 
window opening without difficulty after manually 
removing the bony overhang, with full insertion 
achieved in 7 out of 8 cases (Fig. 3). Partial insertion 
occurred because electrode array advancement was 
stopped as soon as resistance was detected. 


Fig. 3 Manual electrode insertion into the DCA tunnel (Left) 
and post-operative CBCT image with segmented inserted 
electrode (Right). 


DISCUSSION 

The proposed image guided robotic System has 
demonstrated sufficient accuracy for minimally invasive 
cochlear implantation. DCA procedures were performed 
on cadaveric specimens without damage to critical 
anatomical structures, and electrode insertion was 
performed without the need for additional insertion 
tools. Whilst an invasive Standard neurosurgical head 
clamp was used in the presented study, a non-invasive 
head clamp that utilizes inflatable pressure pads is 
currently being developed (Fig. 1). Integrated 
preoperative and intraoperative safety and risk analysis 
with intuitive graphical feedback currently enable the 
surgeon to monitor the Status of the procedure without 
direct visual or video feedback. Additional safety 
mechanisms such as the use of integrated facial nerve 
monitoring and intraoperative CBCT are currently being 
investigated and will be used to increase procedure 
safety and surgeon confidence in the future. 
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INTRODUCTION 

Snake-like robots have the potential to reach deeper 
within the anatomy while reducing patient trauma. To 
help guide these tools, model-based image-guidance can 
be used to fuse live tracking data with preoperative 
imaging in order to visualize a robot’s pose relative to 
the anatomy. In this paper, we introduce a new 
probabilistic filtering algorithm for registering live 
tracking data with a preoperative surface model while 
simultaneously estimating the surface compliance. 

Registration for medical imaging is often performed 
using the iterative closest point method (ICP) [1] or 
with a number of deformable registration methods that 
account for organ deformation [2]. Our probabilistic 
approach overcomes the delay required for offline batch 
Processing by estimating registration parameters online 
with an iterative extended Kalman filter (IEKF). 

Existing methods for in-vivo registration use line-of- 
sight measurements ( e.g ., 3D vision and ränge 
scanning). Our approach altematively uses palpation to 
sense organ shape and compliance {e.g,. [3], [4]). In this 
paper, we extend our prior work [5] by relaxing the 
assumption that the organ stiffness is known. We 
instead simultaneously estimate organ compliance along 
with parameters defming the registration of the robot 
through the use of contact force measurements. 


MATERIALS AND METHODS 

We used a 05mm two-segment snake robot, first 
presented in [7], for experimental validation, see Fig. 1. 
For the experiment, we performed two scans of a 
flexible surface (shown in Fig. 1), by palpating the 
surface while regulating the contact force (first at 5g, 
then at 25g) using intrinsic force estimation [4]. The 
scanned surface has two regions of different stiffness 
(labeled as inner and outer in Fig. 1). Our filtering 
method uses contact force data to infer the registration 
parameters that would best explain the sensor data. 


State Definition: to register the robot to a flexible 
surface, we treat registration as a probabilistic filtering 
task in which we seek to estimate an unknown state 
vector x k that defmes the registration of the robot to the 
surface and which encodes the surface compliance, 


x k = [x 


T 

k, robot 
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T 

k, robot 




where x krobot and 9 k , robot represent the position and 
the yaw-pitch-roll Euler angles of the robot base in the 
coordinate frame of the surface. Vector c k defmes the 
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Fig. 1 For experiments, we tested a 05mm continuum robot 
palpating a flexible surface. 


Measurements: The surface is assumed to be locally 
planar and the contact force is assumed to be normal to 
the surface (we lubricated the surface to ensure this 
assumption holds). In addition to measuring force, the 
robot measures its tip position relative to the base of the 
robot via a magnetic tracking coil (Ascension Tech., 
Trackstar II™) in the robot tip. 

Measurement Model: the measurement model that can 
be used to correct the filter estimate upon observing 
sensor data (force and tip position) is as follows, 

-n T [s(x k )-p]/c(x k ) 1 

%k,tip 

where n is the surface normal, p is a point on the 
(relaxed) surface, and c(x k ) is a function that extracts 
the appropriate surface compliance given the state x k . 
Conceptually, this model is a spring model that equates 
the contact force with the surface deformation divided 
by the compliance. In this formulation, the function, 


h(x k ) = 


s(. x k) R(. x k) x k,tip 


+ t(x k ), 


transforms the estimated position of the robot tip in the 
robot base frame to a position in the coordinate frame of 
the surface. R(x k ) and t(x k ) are the rotation matrix and 
translation vector, respectively, that perform this 
transformation based on the estimated robot base 
position x krobot and the robot base orientation 0 krobot . 


Filter Formulation: Our filter estimates a mean vector 
x k \ k and covariance P k \ k using a static prediction step 
and a measurement update step that uses an IEKF 
update equation [6], 

*i+i = *k|k-i + K[z k - h(x k (fc-i)] - KH k [x k | fc _i - X t ] . 
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Fig. 2 The initial (green) and final (blue) registration of 
contact points. 

function h(x k ) and K is the Kalman gain. Lastly, we 
note that we choose to instantiate a number of these 
filters with different initial guesses 2 0 | 0 . After an 
experiment, the highest likelihood hypothesis is 
reported as the registration result. 

Dataset and Experiment: we split the palpation 
dataset, shown in Fig. 2, into two datasets. The first was 
used to infer a best approximation to the relaxed 
Position of the surface (shown in Fig. 2 as a gray 
plane) 1 . This data was also used to infer the ground truth 
compliances (0.11 mm/g and 0.05 mm/g) for the outer 
and inner regions, respectively. The second dataset was 
used for testing our algorithm. We purposefully 
disturbed the test data with a translation in x and z, 
resulting in a large registration error with respect to the 
surface (shown in Fig. 2). This was done to create a 
challenging registration test for our approach. Finally, 
we iterated through the contact points in the test dataset 
and applied the IEKF update equation for each point. 

RESULTS 

The registration result of our probabilistic filtering 
method, compared to the initial registration, is shown in 
Fig. 2. In this figure, the initially registered contact 
points (in green) are poorly registered, but after the filter 
applies the contact force measurements, the algorithm 
correctly aligns the contact points to the surface such 
that the points are properly displaced “into” the surface 
due to local deformation caused by forceful palpation. 
We note that, during the experiment, the estimate of the 
x dimension of the robot becomes highly correlated 
with the estimate of the surface compliance. This occurs 
because the estimation problem is not observable until 
the surface is palpated at two different forces. In Fig. 3, 
an error plot is shown that depicts the error between the 
estimated compliance and the true compliance of the 
surface for the two (inner and outer) regions. The data 
illustrates the sudden drop in error when the surface is 


1 Determined by assuming the surface is planar and by 
performing an expensive batch estimation problem. 



Fig. 3 The error in the estimated compliance. 


palpated at a second contact force. The final registration 
error in terms of aligning the robot to the a priori 
surface was 0.55mm. 

DISCUSSION 

The contribution of this work is a new method, capable 
of running in real-time, for registering a robot to a 
flexible environment. Unlike other approaches, our 
approach uses probabilistic filtering, which allows us to 
estimate the compliance/stiffness of the environment 
while simultaneously aligning the robot to an organ. 

We note that our experiment was a proof-of-concept 
trial that only evaluated 2D registration (the z and x 
dimensions of the robot). For future work, we will test 
more complicated registration problems. 

The significance of our approach is more apparent as 
it relates to a “chicken-and-egg problem”: if the 
compliance of the environment is known, the expected 
displacement of each contact point into the surface can 
be determined, thus simplifying registration. 
Conversely, if the registration is known, the stiffness 
can be determined by observing the surface 
displacement for a given contact force. We believe that 
solving for both the registration and compliance 
requires a probabilistic approach like the one presented. 
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INTRODU CTION 

Natural orifice surgery is an attractive operative 
approach due to reduced surgical trauma, ‘scarless’ 
cosmesis and in some cases, a more direct route to the 
target operative site. Ongoing concern regarding the 
safety of transluminal access has prompted a shift of 
attention towards more tangible natural orifice 
endoluminal techniques. Surgical platforms being 
explored are currently limited to conventional flexible 
endoscopes, which are known to be problematic due to a 
lack of adequate stability, precision and navigational 
control. 

This purpose of this study is to explore the potential of a 
flexible snake robot to overcome existing technological 
limitations in order to make endoluminal surgery a more 
realisable paradigm. Two emerging clinical applications 
are targeted to demonstrate proof-of-concept; 1) per-oral 

endoscopic cardiomyotomy (POEM) for oesophageal 

12 

achalasia or motility disorders ’ , and 2) per-oral 
endoscopic pyloromyotomy (POEP) for delayed gastric 
emptying, gastric outlet obstruction or infantile 
hypertrophic pyloric Stenosis (IHPS) . 

METHODS 

A flexible snake robot, the i- Snake®, was deployed in a 
cadaveric porcine model setting (Figure 1). This hyper- 
redundant master-slave robot delivers 8 degrees of 
freedom (DOF) across 4 distal articulating joints and 
further 1 DOF via an adjustable semi-rigid neck. The 
four articulating joints are driven by embedded micro 
motors. The hybrid gear-tendon driving mechanism as 
described in provides a compact and stable robotic 
platform design 4 . Extemal diameter of the prototype is 
13mm. Two internal channels of 3.0mm and 2.4mm 
diameter are available for interchangeable focused 
energy devices and air insufflation respectively. 

Five essential procedural steps for both POEM and 
POEP were assessed (Figure 2). 

RESULTS 

Controlled and targeted actuation within the confmed 
workspace of the foregut was attainable in this setting 
(Figure 1). Each procedural step was achieved for both 


POEM and POEP. Submucosal tunnel and myotomy 
lengths of 60 - 80mm were repeatedly performed. 
Submucosal tunnel intubation and navigation was 
identified as the most challenging procedural step due to 
limited visualization and shear forces exerted at the 
mucosal incision site. Mucosal or transmural Perforation 
was not encountered. Although performed in staged 
intervals, estimated time for total procedural completion 
is 30 - 45 minutes. 

DISCUSSION 

This preliminary assessment provides encouraging 
proof-of-concept results supporting the role of a flexible 
snake robot in natural orifice endoluminal surgery. 
Enhanced technical features and overall improved 
ergonomic configuration of the i- Snake® offers 
potentially enabling technology for advanced 
therapeutic endoluminal interventions . This requires 
formal quantification through ex-vivo and in-vivo 
feasibility testing as well as comparative assessment 
against existing platforms. 

Optimisation of the robot for endoluminal surgery might 
include Integration with distal caps to facilitate 
dissection, low friction sheath designs, alternative 
flexible focused energy devices and platform 
miniaturization for indications such as IHPS. The 
workspace of the foregut seems well suited to kinematic 
design of the i-Snake®. Additional clinical indications 
for benign and malignant diseases involving foregut and 
hindgut structures warrant consideration. 
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Fig. 1 Ex-vivo porcine upper gastro-intestinal specimens with 
a) oesophagus detubularised and b) stomach transected in mid- 
coronal plane, demonstrating procedural configurations of the 
i-Snake® for POEM and POEP respectively. Thickness of 
luminal muscle layers were 3mm, 4mm and 10mm for the c) 
oesophagus, d) stomach and pylorus respectively. 



Fig. 2 Five essential procedural steps for POEM and POEP 
were undertaken, defmed by a) creation of a 15 - 20mm 
transverse mucosal incision, bl) & b2) submucosal tunneling 
using the HybridKnife (ERBE Elektromedizin), c) intubation 
of submucosal tunnel, d) myotomy using the HookKnife 
(Olympus) (mucosal tunnel opened for illustrative purpose 
only), and e) extubation of submucosal tunnel in preparation 
for mucosal closure (i.e. with endoclip). 
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INTRODUCTION 

Flexible endoscopy is commonly performed in many 
aspects of medicine, such as ureteroscopy and sinus 
endoscopy. Uretero-renoscopy uses flexible uretero- 
scopes for access and manipulation in the ureter and 
kidney. Uses include stone removal, diagnosis for 
bleeding or malignancy, as well as direct biopsy and 
destruction of malignant lesions. However, existing 
endoscope designs rely on decades-old manual Controls 
for translation, rotation and tip flexion. Because of the 
complexity of movements needed for many applications, 
Integration of a more intuitive and user-friendly control 
System has the potential to greatly enhance the safety, 
efficacy, and efficiency of these instruments. 

Related work. To control the endoscope in flexible 
endoscopy, Johns Hopkins University has designed a 
robot for a flexible endolaryngoscope [1]. Our lab has 
also designed a prototype, which can be remotely 
controlled by a 3D mouse [2]. However, these devices 
are not convenient for securing and removing the 
endoscope and do not have a sterile interface. A 
steerable robot driven by two remote motors through 
Bowden cables is compact but its translation movement 
is manually controlled [3]. Based on the first generation 
of the prototype, our lab designed an ergonomically 
enhanced flexible endoscopy robotic System, including 
a new prototype of the robot, its control System and a 
System assessment Software platform. 

MATERIALS AND METHODS 

The second generation prototype robot has three degrees 
of freedom and is shown in Figure 1. A flexible 
ureteroscope is mounted on the robot so that each motor 
independently manipulates rotation, flexion and 
insertion (translation) of the ureteroscope, respectively. 
Three Maxon motors and three Maxon gear boxes (as in 
the first generation prototype) are used to drive three 
joints. To improve the usability, safety and ergonomic 
performance, the second Version of the prototype has 
focused on the following points: (1) Endoscopist can 
efficiently secure the endoscope; (2) A sterile plastic 
interface between the clean ureteroscope and the robot 
prevents cross contamination; and (3) A rigid annular 
tube connected to the robot is used to support the 
flexible lumen of the endoscope during Operation. 


Annular tube 


Ureteroscope 



Fig. 1 Prototype of second generation of uretoroscopic robot. 


The ergonomic improvement of the securing mechanism 
is shown in Figure 2. A sterile interface is placed on the 
ureteroscope before it is mounted on the robot. One side 
of the sterile interface with a flat shaft is inserted into a 
hole in the drive shaft. The other side of the sterile 
interface is cone shaped and fixed in place by pushing 
tightly into a lock bar. Two snaps located on both sides 
of the lock bar secure it in position. When the snaps are 
pushed, a spring mechanism pulls back the lock bar, 
thus releasing the sterile interface. These improvements 
make it easy to install and uninstall the ureteroscope 
compare to our first prototype, which was presented at 
the Symposium last year [2]. 



Pulley 


Steel Cable 


Lock-bar 


Snap 


Sterile interface 


Ureteroscope Motor 

Fig. 2 Depiction of a 3D model of fast snap-on mechanism to 
secure the ureteroscope and mechanism to drive tip flexion. 
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The architecture of the Overall System is shown in 
Figure 3. The robotic framework is interfaced to a 
Computer through a Galil motor Controller. Interfacing 
Software on the Computer uses the Galil application 
programming interface to control the robotic 
framework. The interfacing Software was implemented 
as a dialog-based MFC application in Visual C++ on the 
Windows operating System. A game Controller, and foot 
pedals serve as the user interface devices that are also 
connected to the Computer. The low cost and light 
j oystick from a game Controller manipulates the tip 
flexi on/extension and the left/right rotation of the 
endoscope. Two foot pedals control the insertion/ 
extraction (translation) motion of the endoscope. The 
input from the user interface devices is converted into 
control signals by the interfacing Software to drive the 
robotic framework. 



Fig. 3 System setup for robotically assisted ureteroscopy. 


SYSTEM ASSESSMENT 

An assessment Software platform has been developed to 
evaluate operative efficiency and accuracy of the robot 
System. As shown in Figure 4, the ureteroscope is 
passed into the ureteroscopy trainer (IdealAnatomic Ine.) 
with colored wires inserted as task targets in the renal 
collecting System. Target wire is seen in inferior calyx 
(Figure 4b, indicated by arrow). In addition, a phantom 
with small colored balls fixed in a hemi-spherical space 
(Figure 4 (c)) has been developed to test the System’ s 
operative efficiency and accuracy to reach selected 
points in space. The assessment is ongoing and will be 
reported at the Conference. 

DISCUSSION 

Robotic control for a flexible ureteroscope has the 
potential to improve procedural efficiency and tip 
placement precision during procedures such as 
exploration of the kidney calyxes. The concept 
presented is also applicable to other endoscopic devices. 
This novel control System for flexible endoscopy offers 
several important advantages including wide 
applicability to endoscope size and type, the ability to 
use commercially available endoscopes without 
redesign, and the ergonomically advantageous user 
interface that can potentially enhance efficacy while 
reducing procedure time, tissue trauma, and instrument 
damage. 



Fig. 4 System assessment platform. (a) Ureteroscopy trainer; 
(b) Screen shot of interfacing Software with ureteroscope in 
the trainer; (c) Hemi-spherical phantom; (d) Screen shot of 
interfacing Software with ureteroscope in hemi-spherical 
phantom. 
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INTRODUCTION 


Computer/robot-assisted surgery (CAS/RAS) holds 
potential for therapy improvements, in terms of 
enhanced procedures accuracy, repeatability, safety, and 
reduced morbidity 1 . In particular, brain surgery still 
poses challenges, and a novel CAS/RAS robotic 
platform for neurosurgery is thus addressed, to be 
integrated in the operating room (OR) by enhancing 
human-robot interaction (Fig. 1A). In such a context, a 
continuous dataflow among surgeon, robotic tools and 
sensors is needed, and an effective human-machine 
interface (HMI) is fundamental to the purpose, 
combining surgeon decision-making with Computer data 
Processing capabilities. A graphical HMI encompassing 
virtual reality (VR) scenes and video streams, and 
allowing physical data input through a graphic user 
interface (UI) was therefore pursued. In particular, a 
virtual Simulator of the robotic platform was 
developed, serving the following purposes. Firstly, it 
can be preoperatively used by the surgeon in order to 
test procedures in silico , so as to gain a better 
comprehension of the envisioned tasks and relevant 
constraints. Indeed, medical images and virtual 
anatomical models (possibly rendering the 
interventional scenario from multiple viewpoints) 
enhance the surgeon intraoperative spatial perception. 
Moreover, the Simulator is expected to improve surgeon 
practice with the robotic platform, and to boost its 
training by shortening the learning phase. Furthermore, 
once implemented in the Simulator the control 
architecture actually used for the platform, it permits to 
better integrate robotic tools with sensor devices, by 
individually testing each component, both through a 
“software-in-the-loop” and a “hardware-in-the-loop” 
configuration. This, in turn, simplifies tuning and 
validation of the chosen control strategy. The developed 
Simulator for robotic NeurosurGERy (SINGER) is 
hereafter described. 
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Fig. 1 (A) CAS/RAS functional scheme. (B) Robotic platform 
components in the lab (HMI not shown). 


MATERIALS AND METHODS 

The interventional module of the platform (Fig. 1B) 
consists of the LightWeight Robot IV+ (Kuka, 
Germany) (LWR), supporting a neuroendoscope (Karl 
Storz, Germany) through a purposely designed handling 
interface. LWR Supports safe human-robot interaction 
and an effective integration in the OR thanks to its 
redundancy, high payload, reduced footprint/weight, full 
joints control, and inherent compliance which permits 
direct handling by the surgeon . The HMI includes a 
Workstation encompassing haptic devices, a manual 
switch for LWR safety lock, a wall display and three 
monitors. The wall display shows the SINGER 3D 
virtual scenario, while the monitors respectively show: 

i) preoperative images and planned tool-path (within 3D 
Slicer image-processing environment; www.slicer.org); 

ii) intraoperative images (acquired by an ultrasound 
System); iii) LWR Status, commands and settings. 
Surgical tasks are preoperatively planned based on 
multi-modal images, and intraoperatively 
monitored/updated by means of the processing module, 
also encompassing the Hybrid Polaris Spectra (Northern 
Digital Ine., Canada) optical tracker (OT). SINGER was 
designed on a flexible Software architecture, to 
smoothly switch between Simulation and real hardware 
Operation. Two virtual and hardware components were 
integrated so far, namely LWR (including the 
neuroendoscope with its interface) and OT. SINGER 
acts as a threefold agent: i) as an emulator, to provide 
each platform component (even a real one) with a 
simulated output of another component; ii) as a 
rendering tool, able to display data received from the 
real hardware; iii) as an interactive 3D UI for the real 
and simulated platforms. Real components 
communication and Simulation were developed using 
the multi-platform block-diagram Matlab/Simulink 
environment. This approach permits to run the same 
control code in Simulation, as well as in soft and in hard 
real-time. In particular, the Simulation, which is handled 
by Simulink itself, neglects the issue of synchronizing 
Simulation time with the real one. Moreover, the soft 
real-time modality is implemented through a customized 
Simulink block that slows down the Simulation, based 
on the differential measurements of the CPU number of 
cycles. Finally, when stringent time synchronization 
with the real components is sought, the hard real-time 
code to be run on a Linux/RTAI machine can be directly 
generated by means of the Simulink Coder Toolbox. 
Such an approach offers the possibility to connect 
simulated and real components in the same Simulink 
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model, and to separately run each component by 
exchanging data through a socket connection. At a low- 
level, a Simulink implementation of the Reflexxes 
motion libraries (www.reflexxes.com) manages LWR 
motion generation and the smooth switching between 
different control functionalities. Moreover, an ad-hoc 
inverse kinematics algorithm was implemented for 
LWR joint-space motion control. In addition, graphic 
rendering of the 3D virtual scenario was developed in 
the V-REP environment (www.coppeliarobotics.com), 
which permits to control each object/model 
individually, and by means of different control entities 
(such as embedded Scripts, add-ons, plugins, remote 
APIs, etc.). In particular, for the application at hand, 
components data transfer with the Simulink control 
module was achieved through the remote API 
framework. Finally, a graphical UI was developed by 
using the Qt framework (www.qt.digia.com); it was 
integrated via a socket-oriented connection with the 
Simulink model that handles the main control code. 

As a preliminary model application for SINGER, a 
point-based calibration procedure 4 between LWR and 
OT was simulated. Such a calibration simultaneously 
finds two transformations matrices (by optimization), 
namely that one between LWR base and OT base, and 
that one between LWR end-effector and the tool 
(tracked by the OT) attached to it. To this end, seven 
Cartesian poses were stored in a Eile and subsequently 
commanded to the soft real-time simulated robot 
Controller. The inverse kinematics algorithm computed 
the corresponding joint positions, which were sent to the 
V-REP robot model by means of the remote APIs. The 
Position (in the OT frame) of the tracked tool rigidly 
connected to the robot end-effector was then sent back 
from the V-REP model to the Controller. It was then 
stored to a Matlab file, together with the pose of the 
LWR end-effector, which is known in the LWR base 
frame. The calibration algorithm was finally run off- 
line, on the recorded data, and the sought transformation 
matrices were obtained by Standard least-square 
minimization. Moreover, in order to simulate the white 
noise which generally affects real OTs, normally 
distributed random numbers were generated at each 
sample time and added to the simulated OT 
measurements. This way, the accuracy of the considered 
calibration algorithm was assessed versus noise 
magnitude. 



Fig.2 Main components of the SINGER 3D virtual scenario. 



Fig. 3 Simulated neuroendoscopic procedure, and LWR 
graphical UI. 


RESULTS 

SINGER 3D virtual scenario is shown in Fig. 2, while 
Fig. 3 reports a simulated neuroendoscopic procedure, 
together with some details of the LWR graphical UI. 
Moreover, Tab. 1 shows the root-mean-square (rms) 
errors obtained from the simulated calibration 
procedure, corresponding to different magnitudes of the 
added noise. 


Table 1 Accuracy results of the simulated calibration 
procedure: rms errors on orientation and position, associated 
with different magnitudes of the assumed noise. 


Noise 

rms Eiror^^^^ 

0 mm 

±1 mm 

±10 mm 

Orientation (deg) 

7.1 • IO' 9 

8.3 • 10- 7 

8.3 • IO 2 

Position (mm) 

2.1 • 10-* 

6.0- 10 1 

6.0 


DISCUSSION 

A virtual Simulator for robotic neurosurgery was 
implemented and preliminary assessed. The developed 
Simulator can be used as an emulator, or as a rendering 
tool augmenting surgeon perception of the operating 
scenario, while also providing an interactive 3D UI for 
testing real and simulated platform components. The 
chosen architecture permits to test the control code both 
in “software-in-the-loop” and “hardware-in-the-loop” 
configurations. Furthermore, the Simulator permits to 
use the same control code in both virtual and real 
scenarios. Accuracy results obtained for the considered 
calibration task are consistent with expectations 
(quantitative comparison between real and simulated 
calibration will be documented in a subsequent work), 
thus supporting the effective usability of the SINGER 
Simulator for robotic platform development, towards 
more relevant neurosurgical procedures. 
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INTRODUCTION 

Trans-oral minimally invasive surgery (MIS) of the 
throat is usually preferred over open surgery because it 
preserves the integrity of the laryngeal framework, 
eliminates visible scars, and promotes faster recovery 
[i]. However, these advantages come at a price of 
limited access, visualization, and depth perception, lack 
of distal dexterity, amplified tremor, limited precision, 
and advanced hand-eye coordination. Hence, 
researchers and surgeons investigated the use of robotic 
assistance [2-4] for MIS of the upper airways. 

All robotic MIS Systems use trans-oral access into 
the throat. Trans-oral access requires the use of a 
laryngoscope, or a curved guiding channel. These 
devices require the mandatory use of general anesthesia. 
They also have been associated with several 
complications such as airway Stenosis, coughing, 
gagging, laryngospasm, hypertension, and tachycardia 
[5]. General anesthesia entails the added costs of an 
Operation room and it has also been associated with 
additional complications such as postoperative delirium, 
myocardial depression, and temporary renal 
dysfunction. 

In this paper we present the approach of trans-nasal 
micro- surgery of the throat. This approach has the 
potential to reduce cost, complications, and patient 
discomfort by avoiding the use of the laryngoscope and 
general anesthesia. We believe this approach paves the 
way for low-cost office-based micro-surgery of the 
upper airways. 

In [6], we presented a robotic System for trans-nasal 
access to the throat and evaluated it on a plastic model 
of the upper airways. The System and its control 
architecture were designed to address fundamental 
challenges such as safe and rapid access into the 
anatomy, enhanced intracorporeal dexterity, force 
sensing, tool delivery, and visualization. 

In this paper, we present a cadaveric pilot study 
demonstrating the feasibility of safe trans-nasal 
insertion, access to the vocal folds and operating the 
end-effector in a realistic surgical scenario. 

MATERIALS AND METHODS 

A System comprising of a 05mm two-segment snake 
robot presented in [6] and a Phantom Omni master 
interface has been used for these experiments. The 
System was mounted to the surgical bed as shown in 


Figure 1. Five cadavers were obtained through the 
Vanderbilt Anatomical Donation Program, 3 females 
and 2 males. After the inspection of the nasal conduit, 
pharynx, and larynx, only one cadaver was selected for 
the study. The other four were not used because the 
internal organs were not well preserved or the nasal 
conduit was obstructed. 

The System can operate in both passive and semi- 
active modes. The passive mode is based on a Standard 
telemanipulation architecture in which the surgeon 
guides the end-effector via a master manipulator while 
observing endoscopic images provided by a fiberscope 
passing through the robot’ s end-effector. The semi- 
active mode is used to reach the surgical site. Düring the 
semi-active mode the surgeon Controls the insertion 
depth while the continuum manipulator autonomously 
conforms with the anatomy using compliance motion 
control [6]. 



Fig. 1 Experimental setup: (1) actuation unit, (2) magnetic 
tracker, (3) bronchoscope, (4) monitor, (5) surgeon. 


The surgical slave is composed of a flexible passive 
stem and two actively bending segments. An additional 
DoF is provided by a linear stage, which Controls the 
insertion of the base of continuum arm. The robotic arm 
has three 01.8 mm lumens that allow for the 
deployment of surgical instruments and end effectors. In 
order to compensate for the unknown shape of the 
passive segment constrained by the nasopharyngeal 
tube, a 00.9 mm electro-magnetic sensor is inserted 
through one of the working channels and attached to the 
base of the two segment continuum robot. A metal 
immune Ascension Tech. trakSTAR 2 flat transmitter 
was placed under the patient ’s head and it provided the 
Position and orientation of the base of the proximal 
actively bending segment. 
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RESULTS 

A picture sequence showing the continuum robot 
descending into the upper airways is shown in Figure 2. 
Under compliant motion control, the continuum 
manipulator was successfully inserted through a 34Fr 
nasopharyngeal rubber tube. The tip of the robot exited 
the tube and entered the larynx right in front of the 
epiglottis. The Controller was then switched to passive 
mode (telemanipulation) and the tip of the continuum 
manipulator was steered on the left in order to pass the 
epiglottis. Using visual information through the 
bronchoscope, the end-effector was first directed toward 
the endolarynx and then inserted further to reach the 
vocal folds. Figure 3 shows the view from the on-board 
fiberscope showing the tip the end-effector in the 
proximity of the voice box. A 00.66 mm needle was 
then deployed and the right vocal fold was punctured, 
simulating injection of Collagen or steroids. 



Fig. 2 Image sequence of the insertion from the auxiliary 
bronchoscope inserted through the second nostril. The tip of 
the robot emerges from the nasopharyngeal tube, bends 
sidwise to avoid the epiglottis, and reaches the larynx. 

DISCUSSION 

The goal of the cadaveric evaluation was to demonstrate 
the feasibility of trans-nasal access into the throat and 
the feasibility of controllable tool tip telemanipulation 
to target injection of vocal folds. Although some of the 
organs were collapsed (i.e. the epiglottis) and the 
surrounding tissues stiff because of the preservation 


process, the end-effector was able to navigate the upper 
airways and reach the vocal folds. Once there, 
instruments such as a fiberscope and a flexible needle 
were deployed and operated. This experiment 
demonstrates the efficacy of our compliant insertion 
algorithm in assisting the surgeon during the 
deployment phase and the flexibility of the control 
architecture in switching between semi-active and 
passive modes in order to navigate the upper airways. 

The ränge of motion and speed of the end-effector 
remain, however, limited due to inadequate travel of the 
actuation pistons. The large deflection of the passive 
stem requires additional actuation compensation. Future 
research will focus on a redesign of the System in order 
to improve ränge of motion and higher end-effector 
speeds. Additional cadaveric experiments will be 
conducted in order to evaluate additional key surgical 
tasks such as biopsy, injection of Collagen for the 
treatment of vocal fold paralysis, and steroid injection. 


Fig. 3 View from the on-board fiberscope showing the 

deployment of a needle. 
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INTRODUCTION 

Upper-extremity tendon transfer surgeries have been 
routinely performed since the 1970s for conditions such 
as stroke, paralysis, spinal muscle atrophy, nerve or 
muscle trauma, and congenital disorders. The surgery 
involves re-routing one or more tendons from an non- 
functioning muscle and directly suturing it to a 
functioning donor muscle in order to partially restore 
hand function [3,4]. 

However, a fundamental aspect of tendon-transfer 
surgery has gone unaddressed. Oftentimes, a single 
donor muscle is directly sutured to multiple recipient 
tendons in order to actuate multiple joints. For example, 
take the case of tendon-transfer surgery for high 
median-ulnar palsy, a severe condition that disables the 
flexor digitorum profundus (FDP) muscle bellies and 
results in an inability to fully close the fmgers, leading 
to weak grasps. 

In order to restore fmger flexion capability, the 
current surgical procedure is to directly suture the FDP 
tendons of all four fmgers to a functioning donor 
muscle, such as the extensor carpi radialis longus 
(ECRL) (see Figs. la and lb). While the direct suture is 
a simple method of at- tachment, it results in directly 
coupling the movement of the distal joints of all four 
fmgers. As a result, the direct suture method prevents 
the fmgers from adapting independently during physical 
interaction tasks such as grasping an object, 
fundamentally impeding post-surgery hand function. 
Specifically, when the hand closes in on an ob- ject 
during the grasping process, if one fmger makes con- 
tact and stops, all the other fmgers will stop before 
making contact since the motion of all the fmgers is 
coupled (see Fig. lb). Thus, the direct- suture attachment 
method re- sults in poor multi-fmger power/enveloping 
grasping ability and may require the patient to use 
unnatural wrist and arm movements to complete the 
grasp. This is a significant issue since the ability to 
perform power grasps is fundamental to the activities of 
daily living, such as when holding objects to feed 
oneself [2]. 

In order to address this fundamental issue in tendon- 
transfer surgery, our group is exploring the use of im- 
planted passive miniature differential mechanisms 1 
called “adaptive coupling mechanisms” to attach the 
donor mus- cle to the recipient tendons (see Figs. lc, ld, 


1 A common application of the differential mechanism is in the au- 

tomobile transmission, where the mechanism enables all four wheels 
to be driven by the same drive shaft and still allow each wheel to 
rotate at differing speeds when accommodating a tum. 


and le). In- spired by the application of these adaptive 
coupling mech- anisms in underactuated robotic hands 
[i], the key idea is that these adaptive coupling 
mechanisms, such as a hier- archical pulley System or 
seesaw mechanism, will enable each digit to continue to 
travel even if another digit actu- ated by the same donor 
muscle is stopped when it makes contact with an 
extemal object, thanks to the rotation of the pulleys or 
the seesaw mechanisms. Initial cadaver experiments and 
Simulation studies we have conducted show that the 
implanted mechanisms enable the fmger joints to adapt 
to the shape of the object during grasping and make 
complete contact [6, 7]. 

In this paper, we present results from a Simulation 
study that show that the adaptive coupling mechanisms 
are able to accommodate for uncertainty that is typical 
in surgery and is typical in a grasping task. Specifically 
in the case of tendon transfer for high ulnar-median 
palsy, surgeons need to accurately choose the tendon 
lengths when attaching the donor muscle to the recipient 
tendons. If the tendon lengths are short by even 5% in 
the conventional procedure, some fmgers would make 
contact pre- maturely during the grasping process, 
exacerbating the weak-grasp problem highlighted 
earlier. Also, since there is always uncertainty in each 
tendon’ s moment arm (or the mechanical advantage the 
tendon has over a joint) since it slides on top of the 
bone, any small Variation in the mo- ment arm would 
also result in premature closing after a conventional 
tendon-transfer surgery. Finally, there will always be 
some error when placing the hand relative to the object 
to be grasped. Small deviations from the object center 
would also result in incomplete or weak grasps after the 
conventional tendon-transfer procedure. For each of the 
three cases, we present results from Simulation that 
show that the proposed procedure using adaptive 
coupling mechanisms is able to accommodate such 
uncertainties. To our knowledge, this is the first time 
that the robustness of post-surgery grasping capability 
has been studied for tendon-transfer surgery. 

MATERIALS AND METHODS 

An open-source biomechanics Simulation platform 
OpenSim [5] was used to evaluate how the proposed 
modification to the high ulnar-median palsy tendon- 
transfer surgery improved the robustness of post- 
surgery hand function. The study focussed on the effect 
of replacing the FDP muscle with the ECRL muscle on 
the flexion of the metacarpophalangeal (MCP) and 
proximal interphalangeal (PIP) joints following the 
conventional and pro- posed tendon-transfer procedures. 
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Fig. 1 (a) Schematic representation of the hand anatomy showing the muscles and the rerouting that occurs in tendon-transfer for high 
ulnar-median palsy. (b) Schematic representation of the conventional procedure. The fmgers do not close in completely on the object 
because of coupled finger movement, (c) Schematic representation of the proposed procedure with the hierarchical pulley System. The 
fmgers close in completely around the object due to adaptive movement enabled by the pulley System (Subfigures (a), (b), and (c) are 
reproduced from [6]). (d) A hierarchical pulley System constructed with off-the-shelf components and implanted in a cadaver forearm. 

(e) An OpenSim biomechanical model of the proposed procedure using seesaw mechanisms. 


The conventional four-tailed procedure was studied 
by adding a weightless body with full freedom of 
movement to the forearm to act as the interface between 
the ECRL muscle and the FDP tendons. The proposed 
procedure was studied by using a seesaw mechanism to 
attach the tendons to ECRL (see Fig. le). Three 
weightless bodies were added to the forearm; one was 
given full freedom of movement, while the others were 
attached to the first body and allowed free rotation 
about the Z axis. The ECRL was attached to the first 
bodys center, and the four FDP tendons were attached to 
the sides of the other two bodies. 

A large sphere was placed near the hand center to 
simulate the grasping of a ball. Soft spheres were added 
to the fmgertips to model the compliant contact between 
the ball and the fmgertips using the Hunt-Crossley 
model. A forward dynamics Simulation of a ball grasp 
was run using each model, providing a two-second 
linear ramp-and-hold excitation profile to the ECRL. 
The joint angles of each digit were measured. Three 
separate tests were conducted in order to measure the 
grasping robustness of the post- surgery hand to the 
uncertainty in the surgical process (in both the 
conventional and proposed procedure) and in the 
grasping task: 1) The tendons that inserted into the four 
fingers were shortened by 5%; 2) The moment arms of 
each of the tendons were varied by 5%; 3) The objects 
Position was varied by 2 cm in three random directions. 
The goal was to see how these variations individually 
affected the total flexion of the MCP and PIP joints in a 
grasping task and observe if the hand was able to still 
make full contact with the target object. 

RESULTS 

Fig. 2 shows that after the proposed procedure, the 
change and ränge of total flexion observed is small even 
if the tendon lengths are short or if there is moment arm 
Variation. In contrast, the change and ränge of the total 
flexion after the conventional procedure for the same 
conditions is large. While not shown in Fig. 2, it was 
noticed that even with Variation in the object’ s relative 
Position with respect to the hand and Variation in 
moment arms and the tendon lengths, the fmgers made 
full contact with the object after the proposed 
procedure. In contrast, after the conventional procedure, 
only one finger made contact for the same conditions, 
resulting in incomplete grasps. 



Fig. 2 The change in and the ränge of total flexion at the MCP 
and PIP joints of all four fmgers measured while grasping. 

DISCUSSION 

The results indicate that the conventional procedure is 
highly sensitive to uncertainty in the surgical process 
and uncertainty in grasping tasks. In each Simulation, 
it was noticed that a hand that underwent the 
conventional pro- cedure was unable to create full 
multifmger power grasps, an important aspect of daily 
living. In contrast, the pro- posed procedure enabled 
the fmgers to accommodate the uncertainty in tendon 
lengths, moment arms, and object location and create 
secure multifmger grasps. The finger forces during the 
grasping process is being investigated. 
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INTRODU CTION 

No-scar surgery is the last frontier in the field of 
minimally invasive surgery [1]. More and more 
surgeries are conducted with a transluminal, 
endoluminal or single port access, in order to reduce 
patient pain, bleeding and hospitalization time. Beside 
these advantages, no-scar surgery greatly increases 
difficulties for the surgeons, because the space for the 
instruments is small and triangulation is difficult to 
perform. Flexible endoscopic Systems are good tools to 
help surgeons in performing these procedures, but 
normally these Systems are very difficult to manipulate 
because of the high number of Degrees Of Freedom 
(DOFs). Robotics shall be a solution, assisting surgeons 
during operations and offering control, precision and 
ergonomics. In this paper we present STRAS (Single 
access and Transluminal Robotic Assistant for 
Surgeons), our new flexible teleoperated robotic System 
intended for endoluminal or transluminal operations. 

MATERIALS AND METHODS 

STRAS is a robotized Version of the Karl Storz 
Anubis® System. Anubis® is composed of three 
flexible sub-systems: a large endoscope and two 
insertable instruments. The endoscope consists of a 
flexible passive shaft (about 35 cm long) connected to 
an articulated distal section (about 22 cm long). This 
section can be deflected in two orthogonal directions 
(noted 2, 3, see Fig. 1) and embeds an endoscopic 
camera. The instruments, which have 3 independent 
DOFs (translation, rotation and deflection, noted 4, 5, 6, 
see Fig. 1), are inserted inside the main endoscope. 
Instruments triangulation is possible thanks to the 
endoscope head design. Actuated instruments, like 
grasper, have another additional DOF for opening / 
closing. Overall, the System has 10 DOFs. 

The robotization process consisted in substituting all 
the manual motion driving with motors. For the 
endoscope deflections, the manual handle knobs were 
replaced by a set of spur and bevel gears, driven by a 
pair of Harmonie Drive servoactuators. The instrument 
actuation mechanisms are enclosed in a cylindrical 
hollow casing connected to the proximal end of the 
flexible instruments. The main endoscope is fixed on a 
mobile platform that allows its translation forward and 
backward (noted 1, see Fig. 1). The robot architecture is 
modular: it is possible to Substitute one instrument 
during an Operation without compromising the 
functioning of the whole System. Every sub- System has 
its own electrical case, while two Adept SMI6 
Controllers drive the motors Controllers and manage the 


communication with the high level interface (a real-time 
Linux PC). 



Fig. 1 System DOFs and main kinematics frames. 

During the normal use of the Anubis® System, two or 
more surgeons are requested to control each part of it. A 
big improvement introduced by STRAS is that a single 
user can teleoperate both instruments and main 
endoscope (the slave System) by means of two haptic 
interfaces (two Force Dimension omega.7, the master 
System). The kinematic architectures of master and 
slave Systems are very different, so an adequate 
mapping between them is necessary. We are proposing 
four different control strategies: 

• Direct axis control: each DOF on the slave System is 
mapped on a particular master System DOF. Several 
customizable mappings are possible, based on the 
user preferences. The default setup links left (resp. 
right) master interface to the left (resp. right) 
instrument. The endoscope control is obtained with 
the left interface after pressing a pedal that 
temporarily disconnects the left instrument. 

• Speed control: the master interfaces positions, with 
respect to a neutral interface position, are interpreted 
as desired speed for the instruments. A force 
feedback effect on the interfaces acts as a spring, 
with the aim to drives the interfaces toward their 
neutral position when the user leaves them. 

• Jacobian position control: instruments Cartesian 
workspaces are directly mapped onto the respective 
haptic interfaces workspaces; the motors velocity 
references are obtained by feeding the Cartesian 
position error to the inverse of the instrument 
jacobian. A force feedback effect proportional to the 
distance between the actual instrument and haptic 
Cartesian positions is applied, with the aim to keep 
track between the interface and the instrument 
movements. 

• Cartesian Inverse Kinematics (IK): as in the 
Jacobian position control, the master interface 
position directly defines the instrument Cartesian 
position. But in this case the motors references are 
computed from the desired instrument tip Cartesian 
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Position by means of an algorithm that we 
specifically developed to obtain a discretized IK 
model. 

We have developed a graphical interface, which allows 
users to select the desired control strategy. The 
underlying high-level application realizes the mapping 
during teleoperation, filters the user input and 
communicates with the low-level Controllers, sending 
the computed motors references (positions or speeds) 
and receiving the slave System Status. 

We have also developed a virtual Simulator of our 
System, with the aim to assess the proposed control 
strategies in several simulated scenarios [2]. In addition 
to the robot kinematic model, we tried to implement 
inside the Simulator some mechanical non-linearities 
effects that are currently present on the manual System, 
with the aim to study their effects on the System control. 

RESULTS 

We have conducted several trials, involving surgeons 
from IRCAD and lab members, with the aim to assess 
the behavior of STRAS according to the chosen 
strategy. These trials included path following tasks in 
simulated environment [2] and testbed pick-and-place 
task in which Cooperation between instruments and 
endoscope movements is mandatory [3]. 



Fig. 2 Path following tasks performed with the virtual 
Simulator. 


In the first trials, conducted with the virtual Simulator 
(see Fig. 2), we tested the Jacobians and the Cartesian 
IK control strategies in two tasks that requested 
respectively precise instrument positioning and fast 
response speed. We found that, in ideal conditions, a 
speed control strategy permits to obtain better accuracy, 
because the input method allows fme variations of the 
actual instruments positions, with the counterpart of 
lower instrument reactivity. On the contrary, the direct 
mapping between haptic interfaces and instruments 
applied on position control strategies permits to obtain a 
faster response of the System to the detriment of 
accuracy. Scaling the user input could easily 
compensate this drawback, but in this case the user 
should reposition the master interfaces with a 
declutching System when he/she reaches the interface 
workspace limit. When non-linearities are added inside 
the Simulation, a position control strategy permits to 
better compensate the non-linear effects. In this case, 
the applied force feedback effect seems to give a better 
control feeling and helps the System control, avoiding 
large user movements that cannot be followed by 
instruments because of non-linearities or motors 
limitations. This result points out the need to have 
correct instruments poses estimation on the real System, 


which could be obtained with an extemal sensor such as 
an EM tracker. 



Fig. 3 Pick-and-place task performed with the STRAS. 


In the pick-and-place task (see Fig. 3) we demonstrated 
that a single user could perform alone a complex task 
that involves the whole slave System, which is 
impossible on the manual System. We executed a 
Cooperation task seizing a ring on a stand with the left 
instrument, grasping it with the right instrument and 
bringing it on a nail. This task was performed in less 
than three minutes with a direct axis control strategy. 
For the moment, Jacobian strategies and Cartesian IK 
are not suitable for the real-time control of the robot, 
due to the lack of a valid measure of instruments 
positions. In both trials depth perception was one of the 
main issues. 

DISCUSSION 

This paper has presented our new robotic System for 
endoluminal and transluminal surgery based on flexible 
shafts and continuum bending sections. After the initial 
setup, the robot can be teleoperated by a single person 
as demonstrated by first laboratory experiments. These 
trials have shown that, from a robotic point of view, the 
control strategy should be chosen according to the task 
that has to be done. From the user point of view, we 
have seen that individuals have very different 
preferences regarding master / slave mapping. Surgeons 
have especially appraised the fastness and the apparent 
fluidity of motion of the System, due to the absence of 
friction feeling. They have also acknowledged the ease 
of use and accuracy of the System. 

Following these good results, authors and surgeons 
from IRCAD have deemed the general design of the 
robotic System very positive, and first trials on animal 
models will be carried out in the coming months. First 
trials also showed small architectural aspects that could 
be improved, but most of the remaining work lies at the 
control level. A dedicated master interface will also be 
developed for optimal control. 
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INTRODUCTION 

In the last twenty years, minimal access surgery (MAS) 
has been widely used in abdominal surgery as it offers 
significant advantages, such as: smaller incision size 
and improved cosmesis, rapid patient recovery and 
reduced medical care, cost and blood loss. However, 
MAS is associated with a number of challenges 
including lack of space and limitations of conventional 
instruments. A significant challenge arises from the 
design of these instruments which have a long, thin 
shaft and are used through small incisions or natural 
body orifices. 



Fig. 1 A schematic representative of cross-sectional tissue 
retraction using MAGS which replaces grasper and hooks 
tissue on to peritoneum. 

In order to overcome the challenges posed by space, 
conventional laparoscopic tools are consistently 
improved to avoid any compromise for shape and 
working environment. A device called “Magnetic 
Anchoring and Guidance System” (MAGS) has been 
developed which effectively replaces conventional 
laparoscopic grasping instruments with two permanent 
magnets as shown in Figure 1. It applies magnets to 
remotely manipulate tissue from outside the abdominal 
wall by generating an interaction force between the two 
magnets [1-4]. The magnets grasp tissue and retract it to 
the abdominal wall. However, using several magnets in 
MAS is potentially troublesome for the purpose and may 
cause magnetic collision during surgery. There is also a 
chance of inadvertently tearing tissue during Operation 
due to such strong magnetic interaction. A way to 
overcome these issues is to apply nano/micro scale iron 
colloid particles to magnetise the tissue and 
simultaneously manipulate them by Controlling an 
external magnetic field [5-7]. This technique can 
dramatically reduce the interactions from magnets to 
provide an approach for tissue manipulation. Moreover, 


the fluidity of colloids mean that they can flow inside 
soft tissue to avoid tissue tearing. Consequently, the 
conventional grasper can be replaced and there will be 
more working space and clearer viewing in surgery. 
Metallic particles have been used in human tissue and 
some clinical practice, however, the responded force 
associated with interaction of nanoparticles and 
magnetic field has not been studied well. The current 
research in Magnetic Resonance Imaging (MRI) has 
indicated that a safe ränge of exposure to magnetic 
fields ranges from 0.2 to 3 Tesla. This paper presents a 
study to optimise the size of particles in a carrier fluid 
by measuring the magnetic force of nanoscale 
ferrofluids (FFs). 


MATERIAL AND METHODS 


FFs are widely used in mechanical and medical 
applications due to its unique fluidity and magnetism. 
FFs are composed of magnetite particles (Fe 3 0 4 ) and a 
carrier fluid. The size of particles is usually nanoscale to 
ensure particles are stable in the fluid and are 
superparamagnetic in nature. Superparamagnetic 
particles magnetise in the presence of a magnetic field. 
The FFs used in this study are commercially available 
products: Liquids research Limited 0 and Chemicell 0 . 
The particles size is lOnm and lOOnm and the quantity 
of FF injected in the tissue is 0.3ml. 



Position of FF injected on a 
porcine liver sample 


Two linear slides with 
programmed motion control 
and force detection 


Injecting System by a 
pushingactuator 


Load cell carries a 
magnet as an indenter 


Fig. 2 The configuration of MagRAT. 


Figure 2 shows the magnetic retraction and adhesion 
tissue (MagRAT) tester designed to observe the effect 
of injection of FF in the tissue and to measure the 
magnetic adhesion force. The MagRAT consists of the 
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two linear slides (Zaber Technologies Inc. @ ) to control 
the motion of the load cell (Omega Engineering Ine. ) 
and the clamping plates. A permanent magnet (dia. 10 
mm) is attached to the load cell to provide the force of 
magnetic adhesion when it is placed near the tissue 
containing FF, and the applied magnetic Feld strengthes 
are 0.3T, 0.5T and 0.6T. FF was injected in the tissue by 
an injection mechanism driven by a linear actuator 
(Firgelli Technologies Inc. @ ). The adhesion force was 
recorded using a Fab VIEW program. 

A 3cm x 3cm porcine liver tissue was placed on the 
clamping plates sandwiched between the top and bottom 
clamping plates. The clamping plates have a square hole 
for injecting the FF and to perform adhesion tests. The 
test Starts with an injection of a pre-defmed quantity of 
the FF into the tissue. A linear actuator attached to the 
syringe moves with a constant speed of lmm/s to pierce 
the needle into the tissue. The depth of injection is 
controlled by advancing the linear slide in 0.5 pm steps. 
Electrical conductivity method was applied to detect the 
contact of the needle to the tissue surface then Starts 
injection process. 

RESULTS AND DISCUSSION 

Figure 3 shows a typical force-time profile. This profile 
shows the motion of retraction in addition to the 
behaviour of tissue relaxation from when the probe and 
tissue are in contact. Here, A is the point of the injection 
of FF into the tissue. AB is the movement of the magnet 
towards the tissue. Point B is the point of maximum 
attraction force when the magnet is just in contact with 
the tissue. C is the point showing the maximum pre-load 
applied at the interface. The magnet is allowed to be in 
contact with the tissue as shown by CD. After a given 
time of contact, the probe retracts to measure the 
maximum adhesion force as point E. 



The results show that adhesion force increases with the 
decrease in particle size. A Micro computed- 
tomography (pCT 100) was used to characterise the FF 
particles distribution in the tissue. Samples were 
prepared for the micro-CT Scanner and generated a 
series of segments. The segments showed distribution of 
iron particles within tissue (Figure 4(a)) at a resolution 
of 11.2pm. An image processing Matlab program was 
used to calculate the density of metallic particles in each 


segment, as shown in Figure 4(b). The density of 
particles in different segments was used as a parameter 
to study the effect magnetic force on the transportation 
of FF particles within tissue. 


Fig. 4 (a) The original segment of CT scan (b) after processing. 

The characterisation of the particles in segments helps 
us to understand the effect of transportation of FF 
within the tissue. The magnetic Feld intensity and 
quantity of fluid injected in the tissue are directly 
proportional to the adhesion force. A comparison of 
particle size shows that the smaller the iron colloid 
particles, the higher the magnetic adhesion force. 

This study investigates the performance of nanoscale 
particles and their feasibility in providing an adhesion 
force for tissue manipulation. This is a novel technique 
and our results here demonstrates that it has promise for 
future clinical use. 
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INTRODU CTION 

Brain shift is one of the most challenging problems in 
image-guided craniotomy procedures [1]. In open brain 
surgeries, the neurosurgeon first removes a part of the 
patienf s cranium, and then opens the dura. Due to 
various factors, the brain tissue deforms from its 
preoperatively imaged shape during the surgery. 
Subsequent brain deformations are caused by the loss of 
cerebrospinal fluid (CSF) and the reduction of 
intracranial pressure, as well as periodic motion due to 
the cardiac pulse and the respiratory cycle. The 
deformations on the brain surface and its interior are 
complex and time-dependent. 

To address the challenge of compensating for brain shift 
during robotic and image-guided surgery, a real-time 
soft tissue tracking Software module is necessary. This 
paper describes a new method for real-time 
quantification of brain shift during craniotomy surgery. 

MATERIALS AND METHODS 

We have designed and implemented a sensor System 
and a processing method for soft tissue deformation 
tracking. Our method uses a pair of calibrated and 
synchronized digital cameras to capture a Stereo video 
stream of the exposed brain surface. It estimates the 
deformation on the visible surface of the brain by 
tracking pairs of texture features on the brain surface. It 
removes outliers during tracking with a new paraboloid 
parametric model. 

The proposed workflow is as follows. Prior to surgery, 
the Stereo System is calibrated to derive the intrinsic 
projection parameters and the Stereo relation between 
the cameras. The calibration parameters are computed 
with the open-source calibration toolkit CALTag [2] 
and the open Computer vision library OpenCV. 

At the beginning of the surgery, once the dura has been 
opened, we detect on the first pair of image frames a set 
of 2D feature points (mostly blood vessels bifurcations) 
on one image and match them to the other image by 
maximizing the Normalized Cross Correlation between 
the template and the search area. 

Next, we perform stereo video tracking on the 
subsequent pairs of image frames. The matching is 
performed from left image to left image and from right 
image to right image (Fig. 1). The matching between 
subsequent points is also computed by maximizing the 


Normalized Cross correlation between the template and 
the search area. A prediction scheme is used to narrow 
the search areas and to accurately match less distinctive 
features. A pyramid hierarchical scheme is used for 
efficiency and accuracy. 

For outlier detection and removal, we use a three-step 
outlier detection scheme: 1) Random Sample Consensus 
algorithm (RANSAC) with fundamental matrix; 2) 
RANSAC with affine transform, and, 3) a new outlier 
detection algorithm based on a parametric paraboloid 
model. The third outlier detection step consists itself of 
four steps: (1) computing the 3D location of each 
tracked point; (2) fitting a paraboloid to the 3D points 
using pseudo-inverse algorithm; (3) measuring the 
distance of each 3D point from the paraboloid, and; (4) 
removing the points whose distance exceeds a 
predefmed threshold. To increase the tracking and 
estimation accuracy, we use sub-pixel matching in all 
feature-matching operations. Based on the 2D stereo 
tracking and the calibrated stereo relation between the 
cameras, we calculate the 3D location of each tracked 
point for each stereo frames with the non-linear 
triangulation algorithm in [3]. To increase the point 
location accuracy, we perform optical lens distortion 
correction in each iteration. The output is a set of 3D 
points and their respective trajectory in time. 

RESULTS 

To evaluate the in- vitro performance of our method, we 
built the stereo tracking System and implemented the 
proposed method. It handles 12 stereo frames per sec. 

We conducted an in- vitro experiment to evaluate the 
performance of our method. A rigid brain model was 
used as a phantom (Fig. 2). The model was moved in 
three directions along straight lines to the total distance 
of 10mm: Lx - along the Cartesian table x axis, Ly - 
along the y axes, and Lxy - along the bisector of lines 
Lx and Ly. Sixty different motions were measured with 
a Cartesian table with an accuracy of 0.001mm. Brain 
surface points were automatically selected and tracked, 
and their 3D movement was estimated. The estimated 
movement was then compared to the ground truth 
movement. This in vitro experiment showed that the 
Overall System tracking accuracy is less than 0.5mm. 

For each line trajectory, a new set of features was 
tracked; every feature had a 21 -position path. To this 3D 
path, we fitted the closest line, choosing the mean as a 
point on the line and the direction of largest variance, 
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obtained from Principal Component Analysis of the 
points in the path, in the line direction. Table 1 shows 
the results of the Stereo tracking positional error of the 
phantom surface with respect to the Cartesian 
displacement table measurements. 

The mean Euclidean error, which is the absolute 
difference between the reconstructed movement and the 
table movement, was 0.14mm±0.48mm. The 
mean radial error, which is the absolute difference 
between the reconstructed point and the line fitted to its 
path, was 0.09mm±0.19mm. The mean rigidity error, 
which compares the reconstructed movement of one 
point to that of the other points, was 0.04mm±0.03mm. 

While we expect the in-vivo errors to be larger, these 
results suggest that our method may be suitable for 
addressing the brain shift problem. 

DISCUSSION 

We have developed a new method for near real-time 
quantification of brain shift during craniotomy surgery. 
First, feature points are detected on Stereo images. Then 
the points are matched to the next Stereo images with 
pyramid and prediction schemes to narrow the search 
areas. Three outlier detection methods are applied, 
including a new parametric model-based outlier 
detection. Finally, 3D points locations are computed and 
tracked while brain shift occurs. An in vitro experiment 
showed a mean accuracy of 0.17mm or better. 

The main contributions of our work are: 1) a new 
texture-based tracking method that does not rely on 
implanted fiducials; 2) a robust tracking point selection 
method that allows accurate tracking; 3) a new outlier 
detection method based on paraboloid model fitting that 
does not require a preoperative image, and; 4) 
implementation and in- vitro validation of the method. 



Fig. 1 Illustration of Stereo and video matching of features. The 
top row shows the first video frame, and the bottom the second. 
Stereo matching is done from left to right frame, and video 
tracking is done between subsequent frames from the same side. 



Fig. 2 Photograph of the experimental setup. The Stereo cameras 
are mounted on a tripod (left) and point to the plastic brain 
model, mounted on a Cartesian displacement table. 


We are currently integrating the 3D tracking results with 
a biomechanical brain model, so as to propagate the 
brain surface deformation to its inner parts. 


Table 1 Stereo tracking positional error of the phantom surface. 
The first column indicates the table line translation. The second 
column is the error type. The third and fourth columns are the 
mean and Standard deviation for each case in mm. 
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Error type 

Mean 

Std 

Lx 

Euclidean error 

0.17 

0.72 

Radial error 

0.09 

0.25 

Rigidity error 

0.04 

0.04 

Ly 

Euclidean error 

0.12 

0.22 

Radial error 

0.09 

0.17 

Rigidity error 

0.04 

0.04 

Lxy 

Euclidean error 

0.12 

0.14 

Radial error 

0.07 

0.07 

Rigidity error 

0.06 

0.02 

Avg. 

Euclidean error 

0.14 

0.48 

Radial error 

0.09 

0.19 

Rigidity error 

0.04 

0.03 
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INTRODUCTION 


This paper examines the development of a low cost 
System for determining position and Orientation of a 
catheter in vivo. The immediate clinical application is 
for pulmonary localisation but the technique is relevant 
across clinical specialities. The System uses planar 
magnetic coils transmitting low frequency magnetic 
fields (<25kHz) and implemented on printed circuit 
board (PCBs), as well as a miniature pick-up coil placed 
at the distal end of a catheter. Low frequency magnetic 
fields pass though tissue with minimal distortion and 
attenuation, hence are well suited for use as position 
detection System. By measuring induced voltage in the 
pick-up coil caused by an array of magnetic sources at 
various frequencies, position and Orientation can be 
determined by solving a non-linear System of equations. 
This method of using planar coils for position detection 
has been demonstrated by Plotkin et al [1]. There are 
also a number of commercially available 
electromagnetic tracking Systems such as Northern 
Digital’ s Aurora and Ascension’ s trakSTAR [2], which 
are used in a ränge of medical procedures. However 
these are typically prohibitively expensive and are also 
susceptible to magnetic interference [2]. The current 
work aims to develop a low-cost, high-speed, ideally 
open-source, alternative to current commercial Systems 
by relying, as much as possible, on analogue signalling 
while maintaining low positional errors. 


MATERIALS AND METHODS 
1. Theory of Operation 

The voltage induced by coil of wire in an alternating 
magnetic field is given by Faradays law of 
electromagnetic induction: 

v = -N— = -N d(B i4) (1) 

dt dt v ; 

where N is the number of tums in the coil, B is the 
magnetic field vector in space, and A is a vector 
representing coil area and Orientation. In Cartesian 
Coordinates, A is given by: 

A = 71 (sin 6 cos cp x + sin 6 sin cp y + cos 6 z ) (2) 
where A the cross-sectional area of the sensor coil, 
while 6 and cp represent the Orientation in spherical 
Coordinates. The magnetic field can be generated from a 
current carrying coil which results in a non-linear vector 
function, R(r). This vector varies spatially with the 
position vector r = { x , y, z } and decreases with distance 
from the coil. Considering current to vary in time as 
I = I 0 sin(u)t) where co is the angular frequency of 
excitation, the resulting peak amplitude of measured 
flux at a given displacement is a non-linear function of 5 
variables, [x, y, z, 6, cp}, given by: 


V = o)NAI\B x sin 6 cos cp + B y sin 6 sin cp + B z cos 6] (3) 

where {B x , B yt B z } are functions of r that represent the 
normalized magnetic field components resulting from a 
current of 1 A flowing through the source coil. 

If multiple coils at different positions are used, a System 
of equations can be formed which can be solved to 
determine position and Orientation. By using different 
excitation frequencies for each coil, the various 
magnetic fields do not interfere with one another and 
can be regarded as independent from each other. 

Plotkin et al [1] demonstrated that 8 planar coils, driven 
with 8 different frequencies are sufficient to accurately 
solve this System of equations. Our design modifies 
their approach by simplifying a number of elements 
such as the detector circuit, and improving the stability 
of the excitation source. 



Fig. 1 Block diagram for the entire System System. An 8 
frequency coil driver circuit generates a magnetic field pattern 
from a planar coil array placed below the patient. The 
magnetic field is detected using the pickup coil sensor and the 
signal is then amplifed and processed. The resulting signals 
are sampled onto the Computer where they are used to 
determine the position of the sensor. 


2. AC Coil Design 

The coils used for the source of the AC magnetic fields 
are formed as planar windings on a printed circuit 
board. This facilitates ease of manufacture and 
reproducibility, i.e., each coil can be made almost 
identical. Square coils were chosen as they result in 
simpler PCB layout. In order to design and simulate the 
coils, the following simple expression for the magnetic 
field due to a current carrying filament was used [3]: 


R(V) _ /VY cxq \ / ac a b 

471 Vlcxal 2 / V |c| \b\ 


) 


, , , , (4) 

471 \\cxa\ 2 J V|c| \b\J ^ J 

where a is a vector in the direction of the filament, b is 
a vector pointing from an observation point P = 
(x, y, z) to the end of the filament, and c is a vector 
pointing from P to the Start of the current carrying 
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filament as seen in Figure 2. The magnetic field due to 
each track on the PCB was calculated using (4) and the 
resultant sum gives the overall magnetic field due to the 
coil. The required magnetic field was calculated using 
(1), by estimating the magnetic field necessary to induce 
a signal of approximately 1mV in amplitude in the 
pickup coil at a distance of 30cm from the source coil. 
This value was chosen to give a reasonable signal to 
noise ratio (SNR) within the field of view (FOV). The 
final coil design has 25 tums and a square side length of 
70mm with a max current of 500mA. 



Fig. 2 (a) 3D representation of a single magnetic field source 
coil. Each coil is printed on a two layer PCB measuring 70mm 
x 70mm, with 25 turns in total. Each track has a width of 
.5mm and a thickness of 70pm. (b) Vectors for calculating the 
magnetic field due to a current carrying wire filament located 
at position P [3]. 

AC field generator Circuit 

In order to generate the desired AC magnetic field, an 
analogue circuit has been designed and developed that 
delivers a constant AC current to the coils. By using a 
constant current design, variations in the coil such as 
resistance due to temperature rise can be neglected. The 
circuit is based on a typical constant current op-amp 
design, with an additional dass B amplifier stage to 
boost the output power. 

3. Pickup coil 

The magnetic field sensor used is a single axis pick-up 
coil. Five hundred turns of 40AWG bifilar enameled 
copper wire (80pm diameter) form the coil which is 
2mm in diameter and 8mm in length. A twisted pair 
cable joins the sensor to the detector circuit. 

4. Detector circuit 

In order to detect small voltages induced in the pickup 
coil, an off-the-shelf Instrumentation amplifier, INA 129 
by Texas Instruments, was used to amplify the signal by 
a factor of 500. Once the signal has been amplified, it is 
then buffered into multiple parallel bandpass filters, 
each of which has a high Q factor (>50) and unity gain, 
to split up each frequency component. Following 
filtering, the amplitude of each voltage is recorded using 
a peak-detector circuit and the phase measured using a 
phase detector. The resulting DC voltages from both the 
peak detector and phase detector can then easily be 
measured and used to calculate the position and 
Orientation. 

5. Computer interface 

The resulting DC voltages are sampled using a National 
Instruments data acquisition (DAQ) card with a 
MATLAB (Mathworks Corp., Natick, Massachusetts) 
interface. The MATLAB Optimization Toolbox is used 
to implement the algorithm to determine the sensor 
position using the measured data. 


RESULTS 


The tracking algorithm, which is based on the 
Levenberg-Marquardt least squares method [4], was 
simulated using MATLAB and its performance in 
tracking an arbitrary trajectory was examined in the 
presence of random noise. The added noise is a 
randomly chosen value in the ränge -2mV to 2mV, 
(signal voltage typically is on the order of hundreds of 
mV). Figure 4 shows the trajectory and the calculated 
position of the sensor. The average position error is 
0.47mm; each calculation took an average of 66ms on a 
Standard laptop Computer with a dual core 2. 5 GHz 
processor. 


Trajectory tracking Simulation 
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Fig. 3 Simulation showing the performance of the detection 
algorithm in the presence of white noise. The magnitude of the 
position relative to the origin is plotted; the error is shown in 
the lower graph. 

DISCUSSION 

The presented System represents a low cost method for 
catheter position determination in vivo. The drive circuit 
and detector utilises entirely analogue components with 
low position errors. PCB coil construction can facilitate 
very low cost. The Outputs of the detector are DC 
voltages which can be easily measured using low cost 
ADCs in place of the DAQ cards used in this initial 
work. The overhead cost of the MATLAB algorithm 
could be removed by implementing the relevant solver 
on a more basic programming platform and this is being 
actively explored. 
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INTRODUCTION 

Cardiovascular diseases (CVDs) are the first cause of 
death globally and atherosclerotic plaque accumulating 
in the arteries is the leading cause of cardiac and 
vascular disorders. An estimated 17.3 million people 
died for CVDs in 2008, representing the 30% of all 
global deaths. Of these deaths, an estimated 7.3 million 
were due to coronary heart disease and 6.2 million were 
due to stroke [1]. Current therapies for acute CVDs aim 
to restore blood flow. The traditional surgical 
techniques for vascular problems are angioplasty and 
bypass grafting. Typical inconveniences of these 
procedures are guidance via fluoroscopy (dangerous for 
patients and operators) and the absence of a real 
removal for plaque and obstruction, which normally are 
by-passed or pushed against the vessel wall. In addition, 
although traditional diagnosis and therapy have 
advanced substantially, research efforts are devoted to 
the development of robotic-aided platforms for 
enhancing steerability of the interventional catheter. 
Recently, some remote catheter navigation Systems have 
been introduced: they are based on steerable cable 
activated sheaths for catheters precise steering and 
coronary interventions, or on a controlled permanent 
magnetic field in which a magnetically equipped 
catheter can align and bend. All these platforms are 
based on catheters, they basically provide just steering 
and not propulsion abilities, and they cannot be adapted 
to wireless or softly wired endo vascular devices. 
Moreover, another significant drawback of the 
mentioned platforms is related to the needs of a high 
radiations dose for the catheter localization for both 
operator and patients. 

In this framework, an innovative methodology, 
based on a computer-aided surgery architecture and 
developed by the authors (MicroVAST platform - Fig. 
1), combines a magnetic locomotion approach for a 
vascular softly tethered endoluminal robot with an 
ultrasound (US) transducer-based algorithm for tracking 
purposes [2], [3]. In this paper, the authors describe an 
innovative ultrasound-based methodology for 
endoluminal robots tracking in cardiovascular 
procedures. The ultrasound-based intra-operative data, 
together with pre-operative multimodal data (lumen 
centreline reconstruction for optimal path defmition), 
would allow for the implementation of an assisted or 
autonomous safe propulsion of a endoluminal device in 
cardiovascular procedures. 



Robotic Amts 


Magnet 


En-dovascular 

Device 


Magnet 


Fig. 1 (a) Schematic representation and (b) real development 
of the MicroVAST platform. 


MATERIALS AND METHODS 

The MicroVAST platform is composed by two 
teleoperated anthropomorphic 6 degrees-of-freedom 
robotic arms: one holding an external permanent magnet 
for the endo vascular device navigation (i.e. master 
robot), and the other one holding the US probe for 
tracking (i.e. slave robot). The US probe (CA430E, 
Esaote, Italy) is a convex array transducer with a 
working frequency between 3.5 and 5 MHz and a radius 
of curvature of 40 mm. The interaction between the 
external magnet and the internal magnet of the 
endovascular device (10 mm in length, 4 mm in 
diameter) is used to propel the softly tethered 
endoluminal robot along the cardiovascular System 
while locomotion is supervised by the ultrasound-based 
methodology. In particular, the US-based control 
algorithm consists of the tracking of the internal device 
by using a combination of optical flow (OF) and 
Segmentation techniques, with the aim to maintain a 
reliable and effective magnetic link along the procedure. 
The algorithm consists of an iterative process composed 
by several phases, described in Fig. 2a- f. At the 
beginning of the procedure, the user manually selects a 
region of interest (ROI) enclosing the endoluminal robot 
(Fig. 2b). Onto the selected ROI, specific Shi-Tomasi 
features of the device are extracted thus deriving the 
centre of mass (CoM) (Fig. 2c). A new predefmed ROI 
is then automatically defmed on the next frame based on 
the previous calculated CoM (Fig. 2d) and a Watershed 
Segmentation is applied for retrieving the exact shape of 
the endoluminal robot (Fig. 2e). Subsequently, thanks to 
a pyramidal Lucas-Kanade OF algorithm, the 
corresponding features are found out and the new CoM 
of the object is calculated (as the arithmetic mean of the 
features positions - Fig. 2f). The described algorithm is 
iterated until the end of the medical procedure, and the 
endoluminal robot CoM and features dynamic state are 
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RESULTS 





Center of US probe 



Fig. 2 (a-f) The US-based control algorithm phases for the 
endoluminal robot tracking and (g-h) the high-level algorithm 
for the cooperative control of master and slave robot; 
endoluminal robot (g) inside and (h) outside a safety window. 


then computed, each frame, for the cooperative control 
of the master and the slave robot. The master and slave 
manipulator are moved together and a high-level 
algorithm compares the position of the endoluminal 
robot CoM with respect to the vertical central line 
(green line in Fig. 2g) of the US frame; if the difference 
exceeds imposed safe limits (safety window, red lines in 
Fig. 2h - device dimension ± 20%), the master robot is 
retrieved in the previous safe position in order to 
maintain or, at least, re-establish a reliable magnetic 
link. In the case of a low number of features (i.e. < 4) 
are detected, the procedure is retrieved to a manual user 
control. In order to validate the US-based control 
algorithm performance in terms of CoM calculation 
accuracy, a dedicated Software was implemented and an 
off-line video has been used for computing the 
difference between the derived CoM and the centre of 
an ellipse-shaped geometry coherently overlaid on the 
object frame (Fig. 3). 



Fig. 3 The tracking error as a difference of the (a) ellipse CoM 
and the (b) CoM extracted from the implemented algorithm. 


Furthermore, the US-based control algorithm was 
qualitatively evaluated in a 3D in-vitro test-bench in 
combination with pre-operative data (fed to the robotic 
arm for the optimal path planning) with the aim to 
implement an autonomous endoluminal robot 
propulsion. The in-vitro Simulator mimics the 
abdominal aortic bifurcation and it was developed by 
using a plexiglass tube (PMMA, 0 25(e)/19(i) mm; with 
an acoustic impedance greater than soft tissue one) with 
a first straight part of 230 mm, bifurcation of 120°, 
length of bifurcations 120 mm and a downslope of 30°. 


The US-based algorithm control loop is processed with 
an average time of 13 ms, which is compatible with real 
time application, and an average number of 12 features 
were extracted in each frame from the segmented 
endoluminal robot. An average error of 0.57 mm 
resulted by the difference of the endoluminal robot CoM 
and the centre of the ellipse-shaped geometry, thus 
demonstrating the effectiveness and accuracy of the 
implemented algorithm. Furthermore, the qualitatively 
in-vitro test established the effectiveness of the 
implemented US-based algorithm in the endoluminal 
robot tracking for the accomplishment, in combination 
with preoperative data for path planning (i.e. derived 
lumen centreline), of a 3D autonomous safe magnetic 
propulsion strategy of an endoluminal robot in 
cardiovascular procedures (Fig. 4). 



Fig. 4 The in-vitro experimental test bench for evaluating the 
platform effectiveness; inset: screenshot of the high-level US- 
based control. 

DISCUSSION 

The US-based intra-operative servo control 
demonstrated to be a feasible and accurate tracking 
System and, combined with pre-operative multimodal 
data, would allow for the implementation of an assisted 
or autonomous locomotion of a softly tethered 
endoluminal robot in cardiovascular procedures. The 
locomotion and tracking strategies will be applied and 
tested in a more realistic pathway, the EVE 
endovascular evaluator (Fain Biomedical, Japan) and in 
ex-vivo and in-vivo conditions. 
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INTRODUCTION 

The direct access to organs during Robot-assisted 
Minimally Invasive Surgery (RMIS) is limited and 
palpation is difficult to be implemented. Therefore, 
there is a need for the development of advanced tactile 
instruments to be used for intra-operative soft tissue 
examination and accurate localization of abnormalities, 
such as tumours. 

Despite the availability of tactile devices for soft 
tissue examination [1] the influence of probing 
behaviour on abnormality detection has not been studied 
yet. Our work underlines the importance of optimally 
chosen speed and load during tactile examination of the 
simulated viscoelastic environment [2]. In this work, we 
study the impact of velocity during manual palpation on 
the detection of hard inclusions in a silicone phantom 
and ex- vivo porcine kidney samples. In addition, 
validation results based on Finite Element (FE) 
simulations are presented. We present the evaluation of 
palpation velocity and show that it is a valuable source 
of information for the development of probing strategies 
and the design of new tactile sensing devices. 

MATERIALS AND METHODS 

Ten participants have been recruited to study the effects 
of velocity during manual palpation. All subjects had at 
least five years of surgical experience. Participants were 
asked to detect hard formations during three sets of 
unidirectional palpation tests using a silicone phantom 
and a porcine kidney sample. Three types of palpation 
velocities were employed during each test - slow, 
natural and fast. Natural velocity is defined as the 
palpation speed, which comes most naturally to 
participant. 

To fabricate the soft phantom, silicone gel RTV6166 
with a ratio of 4:6 and viscosity of 900 mPa-s was used. 
Three hard nodules, diameter of 10 mm, were embedded 
at different depths (3 mm, 5 mm) from the surface of the 
silicone material. To create the nodules, hard silicone 
compound RTV615 with a ratio of 10:1 and 4000 mPa-s 
viscosity was used. 

The experimental Setup, schematically shown in 
Fig.l, was used to measure the exerted Finger pressures 
and palpation trajectories - the latter were used to 
calculate the associated palpation velocities. The 
applied forces were recorded with six-axis force/torque 
sensor MINI 40 (ATI Industrial Automation), whose 
normal force resolution is 0.01 N. A Microsoft Kinect 


sensor (640 x 480 pixel resolution and sample rate of 30 
fps) was used to track the position of the hand in three 
dimensions with the help of the Microsoft Visual C++ 
OpenCV package. The position accuracy for the Kinect 
sensor is 1-2 mm. This value was obtained 
experimentally for an average palpation velocity. 


Camera to track 



Sensor to record applied pressure 


Fig.l Schematic representation of the experimental setup 

Data processing and Statistical analysis was 
implemented in MATLAB 7.12.0 and R statistics 1386 
2.15.2 Software. A three-way analysis of variance 
(ANOVA) was used to test the Statistical significance of 
the palpation factors. The confidence level for null 
hypothesis rejection was set to 95%, for p < 0.05. 

To understand the responses of soft tissue during 
various palpation conditions, FE simulations were 
conducted in ABAQUS 6.10.1. The silicone phantom 
(50 x 30 mm planar block) was modelled based on the 
studies shown in [3]. The size of each finite element 
was set to 1 mm using a quadrilateral element type. The 
modelled nodule had a 10 mm diameter and was 
embedded at two different depths (3 mm, 5 mm) 
corresponding to the experimental setting. A Finger 
during palpation was modelled as a rigid sphere with a 
diameter of 20 mm. The modelled soft tissue surface 
was assumed to be lubricated during palpation 
experiments; therefore, the contact between simulated 
tissue and indenting body was set as frictionless. The 
sphere was indenting the silicone phantom model by 3 
mm at the initial Step of the FE Simulation. 

RESULTS 

Firstly one needs to test the effect of various palpation 
factors, such as palpation material, performance of each 
subject, and applied force and velocity, on the detection 
rate of hard nodules. Among all factors, the effect of the 
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target material, silicone phantom or ex- vivo organ, was 
significantly (F (4 0 o) = 6.23, p < 0.0001) influencing the 
detection rate. This result demonstrates that the 
palpation conditions should be chosen according to the 
given environment. 

This implies that the impact of the traversing 
velocity should be studied for a given type of target 
material (for this case - a silicone phantom). Therefore, 
the ANOVA test is conducted again for the 
experimental data of palpation for the silicone phantom. 
It was detected that the used velocity (F( 4 . 24 ) = 8.97, p < 
0.00001) and the applied force (F (424 ) = 16.67, p < 
0.00001) are significantly influencing the detection rate 
of hard formations. 

The above result Supports the importance of 
correctly chosen force and velocity for soft tissue 
examination. Thus, it is required to study the influence 
of the velocity magnitude. According to experimental 
conditions, the velocity magnitude was subjectively 
defmed by each subject. To exclude individual bias, 
velocity data for all trials was divided into three groups 
using k-means clustering. Three clusters are presented 
in Fig. 2. An overview of detection rates and velocity 
magnitudes for each cluster and correlation with 
experimental results is provided in Table 1. According 
to the velocity measurements, the highest detection rate 
is observed for a slow palpation velocity with a 
magnitude, which does not exceed 125 mm/s. 



Slow Natural Fast 

Fig. 2 Clusters of velocity distnbution tor slow, natural and 
fast palpation velocity 


Table 1 The impact of palpation velocity 


Cluster 

Detection 
Rate % 

Velocity 

Magnitude, 

mm/s 

Correlation with 
Experiments, % 

Slow 

87 

85 - 123 

55 

Natural 

82 

144 - 220 

38 

Fast 

69 

256 - 350 

75 


The experimental measurements of velocity 
magnitude are used in our FE simulations. Fig. 3 
displays the simulated soft tissue model with the 
deformation above the nodule. The detection rate of a 
hard nodule depends on the stress magnitude in the 
target tissue. A subject senses the higher force for the 
unit area when the stress in the material is higher. Fig. 4 
displays the stress magnitude for the indentation contact 
point for two different depths of the location of a hard 
nodule. As expected, a slower speed of palpation 
induces a higher magnitude of stress. A larger depth of 
the nodule leads to a smaller stress response from the 
area above it, especially for fast and natural velocity. 



Fig. 3 Finite dement Simulation of silicone phantom indented 
with a fmgertip above the nodule location (depth 3 mm). 



Velocity: Slow Natural Fast 

r " 1 " + " " T ' . . M l < l t 

Fig. 4 Stress distribution for different depths of nodules - 3 
and 5 mm: (green - slow velocity, red - natural; blue - fast). 

CONCLUSIONS 

This work underlines the importance of correctly chosen 
palpation behaviour for a given palpation environment. 
By means of experimental studies and FE simulations, 
we have demonstrated the impact of velocity during 
manual palpation. We believe that these studies will be 
useful for the creation of advanced probing control 
strategies, and development of tactile sensing methods 
that can be integrated with RMIS. It is planned to carry 
out more extensive studies with a tele-manipulation 
Setup in the future. 

ACKNOLEDGMENT 

The research is funded by the NIHR Biomedical 
Research Centre based at Guy’s and St Thomas' NHS 
Foundation Trust and King's College London. 

REFERENCES 

[1] H. Liu, O. Elhage, P. Dasgupta, B. Challacombe, D. 
Murphy, L. Seneviratne, and K. Althoefer, “A haptic 
probe for soft tissue abnormality identification during 
minimally invasive surgery,” in Advances , 2009, pp. 
417-422. 

[2] J. Zirjakova, K. Althoefer, P. Dasgupta, and T. 
Nanayakkara, “Probing Behaviour for Soft Tissue 
Stiffness Measurement,” in World Congress of 
Endourology , 2012, pp. A20-A21. 

[3] K. Sangpradit, H. Liu, L. D. Seneviratne, and K. 
Althoefer, “Tissue Identification using Inverse Finite 
Element Analysis of Rolling Indentation,” in IEEE 
International Conference on Robotics and Automation 
ICRA, 2009, pp. 1250-1255. 


The Hamlyn Symposium on Medical Robotics (2013) 


42 


Robotic Steering of Cardiac Ultrasound Imaging Catheters 

L.J. Brattain 1,2 , P.M. Loschak 1 , C.M. Tschabrunn 3 , E. Anter 3 , R.D. Howe 1 

1 Harvard School of Engineering and Applied Sciences, Cambridge, MA, USA 

2 MIT Lincoln Laboratory, Lexington, MA, USA 
3 Harvard-Thorndike Electrophysiology Institute, Beth Israel Deaconess Medical Center, 

Harvard Medical School, Boston, MA, USA 
brattain@seas. harvard. edu 


INTRODUCTION 

We have developed a System for steering cardiac 
imaging catheters to automatically visualize heart 
structures and instruments. The goal of this System is to 
improve the efficacy, speed, and safety of imaging 
during catheter-based arrhythmia treatment procedures. 
Many cardiac arrhythmias can be effectively treated by 
radiofrequency (RF) catheter ablation [1], but the 
primary real time imaging modality is x-ray 
fluoroscopy, which does not effectively distinguish soft 
tissues and exposes patients and clinical staff to ionizing 
radiation. Real time visualization of intracardiac 
structures is needed to further improve the efficacy of 
catheter ablation procedures. 

Intracardiac echocardiography (ICE) catheters are 
routinely used during ablation procedures for their high- 
quality ultrasound soft tissue visualization. A major 
challenge during catheter ablation is the lack of acute 
lesion assessment, and ICE visualization of lesion 
growth has the potential to improve acute procedural 
outcomes. However, manual use of ICE requires 
specialized training and maintaining continued 
alignment of the imaging plane is a significant 
challenge. Existing robotic catheter Systems (e.g. 



Fig. 1 ICE catheter actuated degrees of freedom with 
corresponding tip and imaging plane motions 


Hansen Medical, http://www.hansenmedical.com, 
Stereotaxis, http://www.stereotaxis.com, [2]) enable 
teleoperation of catheter Controls and reduce radiation 
exposure, but they do not solve the problem of steering 
an ICE imaging plane towards a desired structure. 

In this study our System manipulates an ICE catheter 
in a water tank to visualize a phantom. Multiple images 
are stitched together to create a volume of the target 
structure. This enablement of real time visualization 
during catheter ablation has potential to facilitate 
improved long-term treatment of cardiac arrhythmias. 

MATERIALS AND METHODS 

ICE catheters have an ultrasound array transducer in the 
tip of a steerable catheter which transmits images to the 
clinician in real time. The catheter handle has four 
actuated degrees of freedom (DOF) that create tip 
motion (Fig. 1). Two bending knobs can be individually 
actuated to achieve pitch (left-right knob), yaw 
(posterior-anterior knob), or a combination of both. The 
catheter handle can be rotated and translated. 

The System consists of a catheter manipulator, 
kinematic and control algorithms, and visualization 
enhancement. The off-the-shelf catheter is articulated by 
DC motors connected to belts for rotation and a lead 
screw for translation. Kinematic algorithms were 
derived to control catheter motions which are difficult to 
achieve manually. Specifying a desired position for the 
ICE catheter tip requires 3 DOF. Since the catheter 
possesses 4 DOF, the extra DOF can be applied to 
constrained Orientation control. By adjusting both knobs 
in conjunction with handle rotation, the imaging plane 
may be rotated about the catheter axis at the tip while 
the physical location of the catheter remains constant. 
Fig. 2(a) shows a Simulation of a catheter with its 
imaging plane rotating about the catheter tip axis, where 
the green lines represent the same vector in each 
imaging plane. Fig. 2(b) shows the joint actuation 
required for achieving this motion. A closed form 
solution for inverse kinematics of the catheter tip based 
on orientation was used to calculate the knob 
adjustments [3]. 
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Fig. 2 (a) Schematic of imaging plane rotation while visualizing CAD model of phantom, (b) example handle inputs for large image 
plane rotation, (c) data for 23° total imaging plane rotation 


A series of images obtained during plane rotation 
across a region may be spatially registered into the 
Cartesian coordinate frame and interpolated into a 
reconstructed 3D volume model of the cardiac cycle. 
Electromagnetic (EM) trackers with RMS position 
errors on the Order of 1.4 mm (trakSTAR, Ascension 
Technology) were mounted to the catheter tip for 
reconstruction. EM tracking is typically used for 
electroanatomic mapping in many interventional 
catheter labs. For testing purposes, this technique was 
applied to image an object of known dimensions that 
was fabricated on a high-resolution 3D printer. The 
phantom was designed with distinctive 3D features that 
would visualize well under ultrasound and be useful for 
catheter tip localization during calibration. 

RESULTS 

Robotic steering data for an ICE catheter is shown in 
Fig. 2(c), where the green lines represent the same 
vector in each imaging plane and the color intensity 
represents the Order in which rotations occurred. The 
imaging plane rotated in 13 steps averaging 1.8° per 
Step for a total of 23°. During rotation the total tip 
motions were constrained to within ±2.0 mm in every 
direction, which is an acceptable error tolerance given 
the accuracy of the EM tracker System and sufficient for 
safe procedural use. Fig. 3 shows 13 image slices 
reconstructed to visualize a region of the phantom. 



Fig. 3 Ultrasound image slices stitched into a volume 


DISCUSSION 

The tests described in this study signify the first use 
known to the authors of applying ICE catheter position 
and orientation kinematics to robotically enhance 
clinicians’ visualization abilities. Fig. 2(c) showed that 
the System was able to rotate the imaging plane using 
the kinematic algorithms without significantly 
displacing the location of the catheter. This could enable 
clinicians to move the ICE catheter to a safe location 
and image structures that are difficult to focus on by 
manual manipulation. Reconstructed volumes can be 
useful in performing diagnoses or lesion assessment 
during ablation. 

This approach can enable other visualization 
techniques, such as allowing a user to choose an angle 
from which to image a target object from a set of 
possible viewing angles, and then automatically steering 
the imaging device to the proper position and 
orientation for monitoring catheter- tissue interactions. 

The capabilities demonstrated in this study are 
meant to be a proof of concept for the use of a robotic 
System to control ICE imaging. Future work will correct 
for System inaccuracies (such as friction, backlash, 
imperfect curvature, etc.), apply safety boundaries based 
on the known locations of surrounding heart structures, 
and use image processing to assist in navigation. 
Kinematic algorithms for imaging can be applied to 
many catheter-based procedures involving visualization 
of various organ Systems via long, thin, flexible imaging 
tools. This robotic System, specifically, has the potential 
to greatly increase clinicians’ visualization capabilities 
while reducing procedure times and training times 
related to mastering the use of ICE. 
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INTRODUCTION 

Remote-controlled catheter navigation Systems have 
attracted a growing interest in recent years for 
endovascular surgery [1]. However, most Systems are 
designed based on master-slave platforms with limited 
automation and collaborative control, without 
considering natural motion pattems and tool interactions 
used in bedside practice. While previous studies on 
catheter kinematics are geared towards endovascular 
skill assessment [2], it is believed that this Information 
can be used in a leam-from-demonstration framework to 
automate certain parts of the surgical procedure while 
reducing the cognitive workload of the operator [3]. 
This study proposes a learning-based approach for 
generating Optimum motion trajectories from multiple 
demonstrations, using a proximal catheter position 
sensor, and automating the catheter motion using a 
robotic driver within a collaborative setting through 
which the operator maintains control of the guidewire. 
Leaming the underlying skill and characteristic motion 
patterns of endovascular navigation to enable semi- 
autonomous robotic catheterisation can help improve 
catheterisation tasks while providing more ergonomic 
operator-robot interactions. 

MATERIALS AND METHODS 

The framework and experimental setup were first 
presented in [4] and are depicted in Fig. 1. The designed 
sensor measures the linear and rotational motion of the 
catheter applied by the operator at the proximal end, 
using two non-contact magnetic rotary encoders and a 
roller based mechanism. Using this Information, motion 
models were learned from multiple demonstrations 
performed by expert and intermediate-level operators 
and different trajectories were extracted for each group. 



Fig. 1 Experimental setup depicting the position measurement 
unit (1) and phantom of the aortic arch (2). 


The procedure was cannulation of the innominate artery 
in an anthropomorphic phantom of a type I aortic arch. 
Trajectories were validated using a robotic catheter 
driver with a similar design to the measurement unit. 

This study expands and further validates the robustness 
of the proposed framework by learning motion models 
from completely inexperienced operators, and 
replicating the generated trajectories on the robotic 
driver. The models are learned from 9 demonstrations of 
the task by 3 inexperienced operators. To create smooth 
trajectories in the presence of inter- and intra-operator 
variability while automating the procedure in stages, the 
linear and radial position signals from the proximal 
sensor are divided into three phases based on the path: 
the descending aorta (phase A), the aortic arch (phase 
B) and cannulation of the innominate artery (phase C). 
The signals across each phase are synchronized using a 
combination of dynamic time warping and linear time 
normalization. Gaussian mixture models (GMM) are 
used to create a probabilistic representation of the phase 
data and smooth trajectories that capture the 
characteristic features of the training data are extracted 
from the models using Gaussian mixture regression 
(GMR) [5]. Trajectories are applied to the robotic driver 
to automate the catheter motion and the cannulation is 
performed 10 times across inexperienced operators. 
Guidewire manipulation is controlled manually by the 
operator, and the System awaits operator input at the end 
of each phase before continuing catheter advancement, 
to allow for collaborative control with the operator. 

Catheter tip positions, measured from a 6 DoF 
electromagnetic position sensor attached to the tip, are 
obtained in the manual training phase as well as the 
robotic phase to validate the performance of the System. 
Different metrics corresponding to catheter tip velocity, 
acceleration, and smoothness of tip motion (the 
accumulative change in slope of the displacement 
signal) are extracted for each phase. Differences 
between these are assessed using nonparametric analysis 
(Wilcoxon rank-sum test, P < 0.05). Further analysis on 
repeatability of catheter movement is also provided. 

RESULTS 

Fig. 2 shows the catheter tip displacement and paths 
from the robotic drive based on the learned model 
against examples of the respective training set, depicting 
much smoother and more homogeneous motion with the 
robot compared to the manual catheterization. 
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Fig. 2 Catheter tip displacement and path for robot-assisted 
catheterisation (1,2) compared to manual catheterisation (3,4). 


Table 2 shows the Standard deviation in catheter tip 
displacement for the robotic drive, as compared to the 
manual approach. The results show that more repeatable 
catheter motion can be achieved with the learned 
trajectories implemented on the robotic catheter driver, 
as opposed to manual catheterisation. 

Table 2 Standard Deviation in Mean Catheter Tip 
Displacement for Robot-assisted vs. Manual Catheterisation 



Robotic STD 

Manual STD 


(mm) 

(mm) 

Descending Aorta 

4.53 

8.96 

Aortic Arch 

3.47 

4.93 

Innominate Artery 

2.23 

3.13 


The significant differences (P < 0.05) between robot- 
assisted catheterisation and the corresponding manual 
training set for the median values of the metrics at each 
procedural phase are shown in Table 1. The results 
depict much smoother catheter motion with the robotic 
approach, as a result of steadier movement with less 
back/forth motion, thereby showing the ability of the 
robot-assisted framework to reduce possible vessel wall 
contact and the risk of embolization in the higher-risk 
areas of the procedure (phases B and C). 


Table 1 Median Values for Statistically Significant 
Differences between Robot-assisted Learned Procedures vs. 
Manual Training Data 


Section 

Metrie 

Robotic 

Manual 

Descending Aorta 

Mean speed (mm/s) 

3.45 

10.44 


Mean accel. (mm/s s ) 

67.94 

278.49 


Smoothness (mm) 

52.69 

158.70 

Aortic Arch 

Mean speed (mm/s) 

2.78 

10.06 


Mean accel. (mm/s s ) 

52.74 

264.42 


Smoothness (mm) 

28.11 

73.04 

Innominate 

Mean speed (mm/s) 

2.99 

11.80 


Mean accel. (mm/s s ) 

56.78 

257.03 


Smoothness (mm) 

18.49 

24.71 


Fig. 3 compares the maximum catheter tip velocities 
and accelerations (median values) between the robot- 
assisted approach and the manual training data at each 
phase over all procedures. The significant differences 
observed show that safer and more stable catheter 
motion at lower speeds and accelerations can be 
achieved by the robot, which can correspond to less 
sudden movements and deflections at the catheter tip 
and a reduced risk of damage to the arterial wall. 



Robotic d Manual 


Fig. 3 Maximum catheter tip speeds and accelerations (median 
values) for robotic-assisted vs. manual training data. Error 
bars represent the interquartile ranges. 


DISCUSSION 

This study extends our previous study using a proximal 
Position sensor to generate learned trajectories from 
multiple demonstrations of a catheterisation task, and 
using these towards robotic automated catheter 
navigation in a collaborative framework whereby the 
operator can maintain control of the guidewire [4]. This 
paper further validates the robustness of the proposed 
approach by learning motion models and encoding the 
underlying structure of movement from completely 
inexperienced operators, and successfully validating this 
through automated catheter navigation using the robotic 
driver. The results show significant improvements in the 
smoothness and stability of catheter tip motion with 
reduced number of movements, which could translate to 
reduced vessel wall contact and damage to the arterial 
wall. The study provides important insights into shared- 
control robotic navigation platforms that utilise the 
bedside catheterisation skills of operators. The approach 
can have further applications toward creating metrics 
for automated skill assessment, by comparing motion 
models obtained from inexperienced operators against 
learned models from experts, thereby also improving 
training and assessment of catheterization skills. 
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INTRODUCTION 

Previously, a static image overlay guidance System has 
been proposed for aiding needle placement interventions 
[1,2]. In this technique, a 2D Computer display image is 
reflected by a semi-transparent mirror, so that the virtual 
image appears floating inside the patient in correct 3D 
Position. This System provides accurate transverse 
image guidance for musculoskeletal interventions of the 
shoulder, hip and spine [3,4]. The previous static 
mounting of the System was either fixed to the CT/MR 
imaging System [1] or on a floor-mounted frame over 
the patient table [2,3]. This mounting required careful 
calibration before each procedure, and was prone to 
misalignments due to structural deformation or 
unintended physical contact with the device. 
Furthermore, the static mount limited the access to the 
patient and excluded clinically relevant ranges of 
motions of the tool and the physician. To overcome 
those limitations, we propose the Mobile Image Overlay 
System (MIOS). The potential applications of MIOS are 
musculoskeletal needle injections, parathyroidectomy, 
percutaneous nephrolithotomy and percutaneous access 
to blood vessel. 

MATERIALS AND METHODS 

MIOS consists of mirror-monitor configuration called as 
viewbox similar to [1,2] and attached to a floor mounted 
articulated counter-balanced System. It is equipped with 
optical markers on the viewbox to measure the pose of 
the image overlay plane during the entire procedure. An 
optical marker (reference marker) is attached to the 
patient’ s skin, near the location of the intervention to 
allow tracking of the virtual image plane pose relative to 
the patient. A set of CT or MR visible fiducial markers 
are placed on the patient before image acquisition. 
These fiducial markers are touched using an optically 
tracked Stylus to register the acquired image with the 
reference marker ’s coordinate System by using 
landmark registration. The System can be used for 
exploration of the image volume by a moving image 
slice overlaid on the patient. The Software will display 
the correct image in real time corresponding to the 
Position of the image overlay plane tracked by the 
tracker w.r.t the patient. After locating the target point 
in the image overlay plane inside the patient, the System 
is firmly locked in this position and the needle trajectory 
is drawn on the image slice in the Software, which gets 
updated in the image overlay plane for needle guidance. 


MIOS overcomes the practical difficulty of accurately 
positioning the static image overlay System [1,2] by the 
motion tracking of the viewbox. 


Workspace analysis: The spine needle injection and 
parathyroidectomy are the two critical procedures which 
defmes the constraints for workspace analysis. 
Workspace analysis defmes the i.) Optimum angle 
between mirror and monitor for maximum viewing 
angle up to which the complete depth of the image 
overlay can be seen through the mirror and Orientation 
of viewbox w.r.t articulated arm; ii.) directions of the 
required motion; iii.) sufficient gap between the 
viewbox and patient while the System is positioned over 
the patient; iv.) oblique rotation of +/- 35° of the image 
overlay plane; v.) sufficient vertical space for needle 
injection. The study was conducted using CAD Software 
“Creo2.0”. Previously proposed 60° System (Fig. la) in 
[1,2] and newly proposed 90° System (Fig. lb) are 
considered for the study. 




Image 

overlay 


image 

overlay 


Fig. la 60° MIOS 


Fig. lb 90° MIOS 


Calibration method: The image overlay plane has to be 
registered to the image reslicing plane in the Software 
and physical position of the patient. Two calibration 
methods were studied: i.) indirect calibration (Fig. 2) 
and ii.) direct calibration (Fig. 3). Indirect calibration is 
the conventional method and uses calibration phantom 
to register the image overlay plane with image reslicing 
plane in the Software. It requires manual alignment of 
the viewbox until the markers in the virtual image 
appears in perfect alignment with the markers on the 
calibration phantom. The direct calibration method is a 
novel approach to register the image overlay plane 
directly to the image reslicing plane in the Software, 
without the needs of physical calibration phantom. The 
tracker determines the pose of the passive markers on 
the viewbox and the virtual markers displayed in the 
image overlay plane. The transformation matrix is 
determined between image overlay plane and viewbox 
from the maker pose measured. During the procedure, 
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the viewbox is continously tracked by the tracker and 
using the computed transformation matrix, the actual 
Position of image overlay plane relative to the patient is 
determined. 


For oblique image display, three concepts were studied: 
i.) five bar link mechanism; ii.) monitor rotation only; 
iii.) viewbox rotation about fixed target axis (Fig. 5). 
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Fig. 2 Indirect calibration Fig. 3 Direct calibration 


RESULTS 

The workspace analysis reported the newly proposed 
90° System (Fig. lb) to be optimal with viewing angle 
up to 35° as compared to 15° for the previously 
proposed 60° System (Fig. la). For oblique images, the 
rotation of the whole viewbox (Fig. 5) was practically 
feasible as compared to five bar link mechanism or 
monitor rotation concept. The requirement of +/- 35° 
oblique rotation was not satisfying with both concepts 
due to reduced user viewing angle and mirror 
interference with the patient. The optimal oblique 
rotation angle is +/- 15° and it will be considered as a 
limitation of the MIOS. Alternatively, the System can be 
rotated about 90° around the vertical axis to view the 
oblique details in the image overlay plane. 

The conceptual view of the MIOS is shown in Fig. 4 
and the first prototype developed is shown in Fig. 6. The 
System shall have four directions of motion as shown in 
Fig. 5: i.) movement of +/-5cm along longitudinal axis; 
ii.) rotation of +/-90 0 about vertical axis of the image 
plane; iii.) oblique rotation of +/-15 0 about the target 
point of needle insertion and iv.) movement of +/-10cm 
along vertical axis to accommodate patients of different 
abdominal size. 


The 90° System has additional benefits compared to the 
60° System: i.) increased clearance between mirror and 
the patient; ii.) reduced System inertia as the System is 
mechanically balanced about the image overlay plane 
and iii.) increased viewing angle which also improves 
the view ability for tracker. 

The direct calibration method is expected to reduce the 
calibration time with simplified calibration steps. Direct 
calibration can be done away from the patient space and 
doesn’t require initial phantom calibration Step. 

DISCUSSION 

Based on successful pre-clinical testing of the static 
image overlay System [3,4], MIOS promises to become 
an even more useful tool for image-guided surgical 
navigations. The mechanical design of the MIOS needs 


to be worked further for precise movement of viewbox 
along longitudinal and patient vertical axes, enable the 
viewbox rotation to view oblique images up to +/- 15°. 
Phantom and cadaver studies need to be performed to 
evaluate the accuracy of direct calibration method and 
to refme the clinical workflow. The initial studies shall 
focus on procedures with negligible effect of 
respiratory. However it is possible to compensate the 
effect of respiration as the patient-attached markers are 
constantly tracked by the tracker. 
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Fig. 5 Ranges of motion 
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Fig. 6 MIOS prototype 
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INTRODUCTION 

The method of hybrid position-force control is currently 
used in industry to control the position of a robot tool, 
as well as the contact forces experienced at the tool-tip 
through interaction with a surface. This paper explores 
the use of such a hybrid control [1] strategy to enforce 
active constraints [2], also known as virtual fixtures [3], 
in both Cartesian and force space. Active constraints are 
a collaborative robot control strategy which can be used 
to guide a human operator towards a predefmed target, 
or prevent them from entering ‘unsafe regions’. 

Active constraints and virtual fixtures have been widely 
used within the literature to limit the position of surgical 
tools to operate only within predefmed “safe regions”, 
in order to avoid damaging close-proximity tissue 
outside this region. However, it would still be possible 
to damage tissue within this safe region by accidentally 
applying excessive force at the tool-tip. A hybrid 
control of both position and force would thus act to 
prevent tissue damage both outside and inside the 
predefmed safe region. However, a conflict between 
position control and force control arises at the regional 
constraint boundary, if the force control contains a 
component directly perpendicular and opposite in 
direction to the regional constraint boundary. This paper 
thus explores an extension of the hybrid force-position 
control concept to the field of surgical robotics, where 
the possible conflict at the regional constraint boundary 
is addressed by a scaled selection matrix. 

MATERIALS AND METHODS 

Hybrid position-force control, as first developed by 
Craig and Raibert[l], assigns a coordinate frame to the 
tool tip, applying position and force control along 
independent, orthogonal axes. The method uses a rigid 
selection matrix that defines each axis either as position 
or force controlled. Hence, for pure force control of all 
axes when the user is in the “safe region”, the selection 
matrix for position control, S p , would contain only 
zeros, and the force control selection matrix, Sß would 
be equal to the idenity matrix through the relationship 

S f — I — Sp 

At a regional constraint boundary, i.e. when requiring 
pure position control along one axis, the designated 
local coordinate frame, {L}, for hybrid control is thus 
defmed as having its z-axis normal to the closest point 
on the boundary, as shown in Fig. 1, leaving the two 
other axes force controlled. 


Regional 

constraint 

region 


Yg 

A Global 

> Z<3 



Fig. 1 Illustration of the frame used for the defmition of 
position and force control. The force controlled y- and x- axes 
are tangent to the regional constraint boundary, and the local 
z-axis is fully position controlled. 


The orthogonal force, i 7 * and position, P*, errors are 
calculated by transforming the selection matrices from 
the local to the global frame, and multiplying them by 
the raw force and position errors, F and P: 

F* = T g _ l S f T g } l F 

p* = t g _ l s v t g 1 l p 

where T G _ L is the transform from the global to the local 
frame. 


The selection matrix for position control will be 
different for two States of either pure force control in the 
“safe region” (zero position control selection matrix), or 
hybrid position-force control when the user penetrates 
the regional constraint (non-zero position selection 
matrix). The change from pure force control to hybrid 
control creates the potential for instability in the control 
System at the regional constraint boundary, as the 
position selection matrix rapidly changes form. Hence, a 
smooth transition region, as seen in Fig. 2, is needed 
between the States of force control and hybrid control as 
the tool-tip approaches a regional constraint, avoiding 
the risk of instability. 

To avoid this instability, the position control selection 
matrix is defmed by the variable a for position control 
along the local z-axis 



0 

0 
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II 
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0 

0 

.0 

0 
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The transition between the two States of the position 
control matrix is then implemented as a function of 
proximity, d, to the regional constraint, where pure 
force control along all axes is implemented when 
d>d min ., and hybrid position-force control for d < 0. 
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Within the transition region, the position selection 
matrix varies with a defined as: 


a = 


1 - 


0 

df d m in 
1 


d > d 


min 


0 d ^ d m in 

d < 0 



Fig. 2 Illustration of the control transition region. To ensure 
orthogonaity between position and force control at the 
regional constraint boundary, the transition region attenuates 
the component of the force control normal to the regional 
constraint surface. 


To validate the proposed active constraint control 
architecture, a teleoperated System was assembled with 
a constrained slave - a Stäubli TX90 robotic arm 
(Stäubli Ltd.) - and a haptic master - a Phantom Omni 
(Sensable Technologies, Ine.) - controlled, using the 
Orocos Real-Time Toolkit [4] and the Stäubli LLI as 
described in [5]. A regional constraint was projected 
onto a soft, flat surface which oscillated at 0.3 Hz 
orthogonal to the constraint boundary. The regional 
constraint was marked with a 10 mm thick strip along 
the oscillating surface, and the test subjects were asked 
to follow this strip along the length of the test surface. 
The test evaluated the accuracy of the position and 
contact force under the four control methods; hybrid 
position-force control, position control, force control, 
and no active constraint Support (i.e. unconstrained 
teleoperation with no haptic feedback). 



Fig. 3 Picture of the hardware setup on the left, with the 
Phantom Omni master in the foreground and the Stäubli TX90 
slave in the background. The picture on the right shows the 
setup of the oscillating surface, seen as a grey foam block on 
the bench under the Stäubli TX90 tool. 


RESULTS 

Performance of the different control methods was 
evaluated based on positional root mean square (RMS) 
error and contact force error. The results are shown in 
Fig.4 and Fig. 5. 
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Fig. 4 Comparison of RMS position error distribution for tests 
using the four active constraint control methods. 
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Fig. 5 Comparison of RMS force error magnitude distribution 
for tests using the four active constraint control methods. 

DISCUSSION 

From Fig. 4 and Fig. 5, it can be seen that there was 
significant improvement in RMS positional and force 
errors when implementing active constraints over non- 
constrained cases, with the hybrid Controller also 
performing as well or better with regards to force and 
position constraints, when compared to their single 
mode control equivalents. This Supports the claim that 
hybrid position-force control can be implemented to 
simultaneously reduce both the positional error and the 
excessive tool/tissue interaction force in robotic assisted 
surgery. The primary focus of future work will be 
testing the proposed Controller when extemal forces and 
regional constraints are the product of less structured 
environments. 
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INTRODUCTION 

Transurethral laser surgery of the prostate for the 
treatment of benign prostatic hyperplasia (BPH) is 
traditionally performed via the resectoscope (Fig. 1), a 
telescopic straight instrument that allows only 
translation and rotation along/about its longitudinal axis, 
thus dramatically limiting dexterity and tactile feedback 
at the tip. These issues represent a limitation since, in 
most procedures, the laser tip must be in contact with 
prostatic tissue [1]. In particular, the closer the contact, 
the better will be the distribution of laser energy and 
consequently tissue ablation will be more homogeneous; 
homogeneous ablation avoids formation of craters that 
can entrap tissue fragments, debris and perfusion liquid 
that partially absorb and scatter the incident light 
thereby hindering laser effect [2]. A number of attempts 
to achieve force and tactile sensing in minimally 
invasive surgery (MIS) are reported in literature [3] -[7]. 
Different actuation strategies for flexible, continuum 
and snake-like robots for MIS can be found in [8]-[ll]. 
In this paper, we present the design and preliminary 
evaluation of a novel, miniaturized, flexible robotic tool 
for laser assisted transurethral surgery of BPH. The aim 
of the System is to allow a more homogeneous tissue 
ablation, in order to speed up surgical procedure 
duration and recovery time. The design includes an 
optical Tiber based tactile sensor System to monitor 
contact between prostatic tissue and laser, and a cable 
driven actuation mechanism to steer the tip of the laser. 
Preliminary study demonstrated the theoretical ability of 
the System to sense contact forces up to 0.4 mN and to 
steer the tip in 3D of about ± 10 °. 


Laser Resectoscope 



Fig. 1 OverView of prostate laser ablation procedure. The 
resectoscope is inserted through the urethra to access the 
prostate. 



Fig. 2 CAD model of the ASTRO System and detail of the 


section. 

MATERIALS AND METHODS 

This paper presents the design of a novel, miniaturized, 
flexible robotic tool for transurethral laser surgery of 
BPH: ASTRO (Actuated and Sensorised Tool for laseR 
assisted surgery of the prOstate), shown in Fig. 2. 
ASTRO consists of a flexible multilumen catheter with 
an outer diameter of 2 mm (0 Ext ), in order to be inserted 
in the working channel of a commercial resectoscope 
(Karl Storz 27042LV: 0= 2.5 mm), and a central 
working channel of 1 mm (® In ), which will allow the 
insertion of a commercial laser fiber for the ablation of 
the prostate. Moreover, ASTRO has six small lumens 
(O in, 6= 0.21 mm), around the periphery forming the 
vertices of a hexagon, to integrate sensors for contact 
detection between laser and prostatic tissue, and cables 
for actuation. Three Fiber Bragg Gräting (FBG) sensors 
(SmartFBG, Smart Fibres) are integrated in a way to 
allow 3 DOF contact force sensing at the tip of the laser. 
The multilumen is in black opaque polyamide (PA 12) in 
order to ensure optical isolation between the laser and 
the FBG sensors. Young's modulus of PA 12 is 1.1 GPa, 
thus allowing the multilumen to serve both as a 
mechanical continuum for strain transmission between 
laser and sensors (that have a coating of polyimide with 
a Young’s modulus of 2.5 GPa), but also to be flexible 
enough to be bent by cables. The melting point of PA 12 
(178 °C) is compatible with the application since 
temperature of prostatic tissue vaporization is around 
100 °C. Moreover, the temperature drops dramatically 
as soon as we move away from the ablation site, 
because of sahne solution's continuous flow inside the 
resectoscope sheath. FBG technology has been chosen 
because of its small dimensions (0=0.18 mm), 
electromagnetic interference immunity, sterilizability, 


The Hamlyn Symposium on Medical Robotics (2013) 


51 



and biocompatibility. These sensors are affected by 
cross-sensitivity of temperature and strain, the shift in 
Bragg wavelength Ak being expressed as: 

AA = k £ s + k T AT (1) 

where k s is the strain sensitivity, s is the strain, k T is the 
temperature sensitivity and AT is the Variation of 
temperature. Redundant FBG sensors are used to 
compensate for errors due to temperature effects, such 
as in [5]. Thanks to the sahne solution’ s continuous 
flow, we can assume that temperature variations will not 
be high at the sensor level: three FBG sensors are 
embedded parallel within ASTRO at 120° from one 
another, at 2 cm away from the tip of the laser. In this 
way, sensors experience the same AT \ but different 
strain in terms of compression and tension, and 
therefore the temperature compensation can be 
performed easily. Three nylon cables (0=0.15 mm), 
that will be actuated by motors, are fixed at the tip of 
ASTRO and used for laser steering. FEA simulations 
were performed in order to assess the adequacy of 
sensor and actuation strategy choices. 

RESULTS AND DISCUSSION 

The best positioning of the FBG sensors, along with the 
minimum detectable ASTRO contact force, were 
estimated with FEA. Iterations of decreasing contact 
force, applied at the ASTRO tip, versus computed strain 
measured at the FBG level were performed: the results 
of the simulations are reported in Fig. 3 (a). Since the 
FBG strain resolution is given as 0.4 pa, the minimum 
force that can be measured, positioning the FBG sensors 
at 2 cm from the laser tip, resulted in 0.4 mN, as shown 
in Fig. 3 (b). As regards actuation, the necessary pulling 
force, to be applied on a cable by a motor, for steering 
the laser to reach a proper contact with prostatic tissue, 
was estimated with FEA. Since the diameter of the 
prostatic urethra in a healthy prostate is around 6 mm 
and the ASTRO tip is 2 cm length (shown respectively 
as d and / in Fig. 4 (a)), a tip bending of ± 10 ° (6) was 
estimated to be sufficient to have a proper contact in all 
cases. Thus, the required pulling force resulted in 12 N, 
as shown in Fig. 4 (b). Future works will be the 
fabrication and testing. In conclusion, the proposed 
System could allow a homogeneous treatment of BPH, 
shortening the operative and recovery time. 
Furthermore, it is compatible with current commercial 
instrumentations (resectopes and laser Tibers). 
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Fig. 3 Evaluation of minimum detectable contact force of 
ASTRO System: a) contact force applied versus computed 
strain measured at the FBG level, b) FEA: Von Mises stress in 
the section of ASTRO System with 0.4 mN contact force 
applied. 
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Fig. 4 ASTRO tip steering: a) estimation of necessary tip 
bending, b) FEA: tip steering with 12 N cable pulling force. 
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INTRODUCTION 

Point-based registration using surface fiducial markers 
is the most commonly used method for patient-image 
registration since the surface markers are not invasive 
and are easily identified in the patient and image [1, 2]. 
However, registration accuracy gradually decreases 
proportionally to the distance from the surface where 
the markers are attached [3, 4]. Since surgical targets 
are located within the patient body, high accuracy is 
required in regions relatively far from the skin. Fiducial 
registration error (FRE) is automatically calculated by 
the navigation System, but the FRE cannot guarantee the 
real accuracy of the target registration. To decrease 
target registration error (TRE) around the target rather 
than FRE around the skin markers, additional points 
should be incorporated for the registration near the 
target organ. In this study, we propose a method that 
uses an anatomical landmark around the target as an 
additional fiducial point for the registration to decrease 
TRE. 

MATERIALS AND METHODS 

To take an anatomical landmark as a reference point, an 
ultrasound System (ACUSON X300, Siemens, 
Germany) was used. An optical position sensor (Polaris 
Vicra, NDI, Waterloo, Canada) was utilized to detect 
and track the surgical tool and ultrasound probe. 3D 
Slicer (Brigham and Women’s Hospital, Boston, USA), 
which is free, open-source Software, was used as basic 
platform Software in this research. 

Figure 1 shows the whole System configuration for 
patient-image registration using deep anatomical 
landmarks. Surface fiducial markers are markers that is 
attached to the patient skin and identified in the medical 
images for registration. Reference marker is a set of 
infrared reflecting markers that are attached to the 
patient to compensate patient movement. Anatomical 
landmarks could be sharp comers or edges of bone, 
vessel, or any specific point of Organs in the body that 
can play a role as markers for the registration. 

Figure 2 represents the suggested method to acquire 
an additional fiducial point using ultrasonography for 
patient-image registration. Optical sensors were 
attached to the ultrasound probe, and the optical 
tracking System was used to track the position of the 
probe. By calibration between the ultrasound image and 
the ultrasound probe, the position of the ultrasound 
image can be transformed to the position of the optical 


tracking System. Additional fiducial points were taken 
from the ultrasound images using the optical position 
sensor that recognizes the position of the ultrasound 
probe. After transformations between the anatomical 

landmarks and the ultrasound probe ( * 7 in Fig. 2), the 
anatomical landmarks were treated as the markers for 
registration, as with surface markers. 

To evaluate the improvement in the accuracy of the 
proposed method, a phantom experiment was designed. 
An acrylic box filled with pure water was used with the 
ultrasound System. The tracking System tracked both the 
phantom and the ultrasound probe. The ultrasound 
probe detected landmarks inside the phantom. With the 
aforementioned transformations, which were calculated 
during the calibration, landmarks were used as fiducial 
markers for the registration. Based on the fiducial 
points, the paired point registration method was 
performed between a reconstructed image of the 
computed tomography (CT) volume data and the 
phantom. 

Two experiments were performed. First, seven 
fiducial makers attached to the surface of the phantom 
were used for the registration. Second, in addition to the 
seven fiducial markers attached to the surface of the 
phantom, one deeply seated landmark was additionally 
used for the registration. We measured the FRE and 
TRE in each experiment. TREs were measured using 
three points of the landmarks inside the phantom except 
for the point that was used for the depth fiducial marker 
in the registration. 
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Fig. 1 System configuration of the proposed method 
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Fig.2 The proposed method to acquire deep fiducial positions 
for patient to image registration 


RESULTS 

The registration accuracy with and without an 
anatomical landmark was examined by using seven 
surface markers and one additional anatomical 
landmark. Figure 3 shows the FREs and TREs. FREs 
without and with an anatomical landmark were 0.76 ± 
0.12 mm and 2.13 ± 0.37 mm. TREs without and with 
the anatomical landmark were 1.46 ± 0.31 mm and 1.27 
± 0.39 mm for TRE1, 1.69 ± 0.41 mm and 1.89 ± 0.73 
mm for TRE2, 1.24 ± 0.47 mm and 0.95 ± 0.47 mm for 
TRE3, and finally 1.65 ± 0.74 mm and 1.04 ± 0.53 mm 
for TRE4. Except for TRE2, the additional use of the 
anatomical landmark caused an increase in FRE, but a 
meaningful decrease was observed in TRE. 

2.5 
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Fig. 3 Registration error with and without an anatomical 
landmark 


A POSSIBLE APPLICATION 

The proposed System could find its clinical application 
in orthopedic surgeries in terms of the depth of the 
surgical target, and the availability of anatomical 
landmarks. Specifically, the proposed System could 
assist surgical navigation for epiduroscopic (EDS) 
procedure, which is one of our ongoing research works. 
Our EDS navigation System tracks positions of a 
surgical tool (catheter) by electromagnetic tracking 
System (EMTS) and a patient body by optical tracking 
System (OTS) rather than both of them by only EMTS. 
The additional use of the OTS aims at utilizing 
relatively high tracking accuracy of the OTS in both the 
registration and the navigation. In the patient-image 
registration of our EDS navigation System, if we can 
include deeply seated landmarks obtained intra- 


operatively by the ultrasound System as well as surface 
markers in the fiducial point set, the registration error 
would be reduced, and in turn, the navigation accuracy 
is expected to be highly improved. 
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Fig. 4 Surgical navigation System for epiduroscopic procedure 


DISCUSSION AND CONCLUSION 

We found out that TRE can be reduced by 
simultaneously using the deeply seated anatomical 
landmark as well as skin-affixed markers. In the 
phantom experiment, we confirmed that the target 
registration accuracy was improved when the additional 
points were used. Although the anatomical landmark 
affects the improvement of the accuracy, we have to 
compromise and use the limited number of anatomical 
landmarks because, in general, they are very few around 
the target. In addition, if the error caused by the 
distortion of ultrasound image and insufficient accuracy 
of the ultrasound probe calibration is reduced, patient- 
image registration will be more accurate, and TRE will 
be more decreased in the future. Although the FRE was 
increased due to the distortion and insufficient 
calibration accuracy, the reduced TRE shows the 
feasibility of the method. 
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INTRODUCTION 

The human larynx can be affected by laryngeal cancer 
and tumors. Particularly, malignant or benign 
pathologies of the vocal folds influence the patient’ s life 
significantly because swallowing, breathing, and voice 
production are disrupted. In case of surgical treatment, 
phonomicrosurgery is considered as state-of-the-art 
technique for preservation and improvement of 
functionality. Operative preparation involves 
laryngoscopy in anaesthized patients for direct laryngeal 
exposure. Intra-operatively, surgeons employ operating 
microscopes for transoral examination and observation 
of surgical procedures. Tissue manipulation on the 
vibratory edge of the vocal fold is performed manually 
with long microsurgical instruments (e. g. forceps and 
scissors). Less soft tissue trauma and improved cutting 
accuracy is expected from extending the conventional 
optical setup with laser sources and manually operated 
micromanipulators for beam deflection. This 
configuration enables simultaneous control and 
observation of the laser incision. However, the quality 
of surgical training and the surgeon’s dexterity 
determine the postoperative outcome due to the non- 
ergonomic design of the current setup. 

In order to overcome these limitations, robots have 
been introduced in different medical applications. With 
respect to laryngeal surgery, robot-assisted access has 
been established in transoral robotic surgery (TORS). 
The commercial da Vinci System (Intuitive Surgical, 
Ine.) has successfully been used in TORS of laryngeal 
lesions by utilizing microinstruments [1] and C0 2 
laser [2]. Haifeng et al. developed a master-slave serial 
robot with 7 degrees of freedom (DoF) for this specific 
use [3]. Simaan and Rivera-Serrano et al. introduced 
highly articulated, snake-like robots which have been 
adapted from prior applications in laparoscopic and 
thoracic surgery [4, 5]. Furthermore, Mattos et al. 
proposed a teleoperated robotic System for visual 
planning and safe laser cutting [6]. Despite recent 
developments in robot-assisted laryngeal (laser) surgery, 
these procedures still have restrictions. For instance, 
serial manipulators equipped with conventional 
microinstruments require a direct line-of-sight to the 
vocal fold which cannot be guaranteed for all patients. 
In this context, flexible robots perform appropriately but 
their design prevents a high-density integration in the 
distal end. Besides, both kinematics require large 
external mechanisms. This paper reports on the first 
steps towards an endoscopic System for laser-assisted 
phonomicrosurgery. The objective of the proposed 
device is to overcome the aforementioned limitations by 


means of minimizing the working distance between 
laser manipulator and tissue surface from 400 mm for 
conventional Systems to approx. 20 mm. The proposed 
approach decreases the distance by inserting a laser 
delivering endoscopic device into the patient and its 
subsequent feeding closed to the surgical site in the 
larynx. The distal integration of a beam deflecting 
microrobot (developed by MiNaRoB, FEMTO-ST, 
France) enables to ablate pre-planned laser incision 
paths on the tissue surface. Additional vision devices 
and kinematic mechanisms in the distal end extend the 
surgeon’s view and facilitate pre-positioning with 
respect to the lesion. In this regard, preliminary design, 
actuation, and control principles are presented. 

MATERIALS AND METHODS 

Initially, design constraints and specifications are 
determined by analyzing medical and technical aspects 
of laryngeal surgery. Device dimensioning is therefore 
influenced by the anatomic nature of the upper airway. 
A transoral access to the surgical site requires an 
insertion of the device through the mouth, subsequent 
feeding of the distal part through pharynx and 
hypopharynx to reach the vocal fold in the larynx. The 
involved anatomy is characterized by a long and narrow 
soft tissue tube with a patient-dependent cross-section 
of 15 -21 mm. Thus, prospective designs of the 
endoscopic shaft should not exceed a maximum 
diameter of 20 mm to enable gentle feeding. 

Further on, current surgical procedures comprise 
Suspension retraction to elevate the epiglottis and 
straight laryngoscopes for vocal fold exposure. This 
methodology often causes pre- and postoperative 
complications due to overstretching of the patient’ s 
neck. The proposed concept should obviate the need for 
direct line-of-sight by integrating the entity of required 
devices for visualization and manipulation into the 
distal end. In order to facilitate the insertion, the shaft 
should be pre-curved, comparable to curved blade 
laryngoscopes. According to the technical 
specifications, the microrobot deflects the laser beam in 
a ränge of ±25° and the Stereo camera setup shows a 
restricted field of view of approx. 65°. With regard to 
these limitations, additional DoF of the distal end are 
required for pre-positioning and visual examination of 
the surgical site. Within this context, a mechanical 
compensation of respiratory movements is not required 
due to a vision-based tissue tracking. Therefore, the 
minimum joint angle for Orientation adjustment in two 
axes and the working distance between vocal fold and 
distal end are determined to ± 20° and approx. 20 mm, 
respectively. 


The Hamlyn Symposium on Medical Robotics (2013) 


55 



This configuration should facilitate Stereo vision 
feedback to the surgeon and three-dimensional tissue 
recovery for augmented reality. 


RESULTS 

The proposed distal end design (see Figure 1) is 
characterized by high-density Integration. Current 
components for vision applications include a 
stereoscopic setup with chip-on-the-tip cameras 
showing a resolution of 720 x 580 pixels and a single or 
multiple Tiber based whitelight or narrow band imaging 
(NBI) illumination. Moreover, image fibers are 
connected to proximally located high- speed cameras for 
vision-based control (visual servoing) of the beam 
deflecting microrobot in order to ensure a precise laser 
cutting path. Furthermore, a tool channel with a 
diameter of 3 - 4 mm is integrated to feed additional 
instruments to the surgical site. The alignment of the 
components is optimized regarding the central position 
of the microrobot and an appropriate camera view with 
small baseline. For protection and assembling purposes, 
distal parts are mounted to a custom support that can be 
inserted and fixed to the housing. Furthermore, the 
support connects the distal end to a kinematic module 
satisfying the aforementioned specifications. 
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Fig. 1 CAD model showing distal end, kinematic module 
protected with bellow and transition to shaft (outer 
diameter 20 mm) 


allow overcurrent sensing and encoder analysis. High- 
level control is established by an ethernet connection 
between the embedded device and a main Computer 
which runs a Robot Operating System environment 
(www.ros.org). 

Fig. 2 CAD models of kinematic module: parallel (1.) 
and serial (without prismatic joint) (r.) configuration 

DISCUSSION 

In this contribution, first developments of an endoscopic 
device for laser-assisted phonomicrosurgery are 
presented. Primarily, state -of-the-art Systems are 
discussed and the new device is motivated by 
emphasizing clinical benefits for patient and surgeon. 
Despite not having finalized the mechatronic System 
design, a significant enhancement is expected by 
integrating a beam deflecting microrobot and vision 
devices directly into the distal end. This obviates the 
need for direct laryngoscopy and therefore provides a 
gentle intervention. Furthermore, the integration of 
imaging components allows surgeons to employ Stereo 
vision, surface recovery, and tracking for safe laser 
guidance and visual feedback. In the next step, 
prototypes will be manufactured to evaluate feasibility 
and performance in experimental studies. 
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This module enables at least two rotational and one 
translational DoF. Two different kinematic chains are 
considered appropriate (see Figure 2). On the one hand, 
a three-legged parallel robot (e. g. 3-PRS or 3-PSR) 
allows reducing the Overall size but requires push/pull 
rods or precision engineered telescopic legs. On the 
other hand, a serial kinematics with three DoF 
comprising a universal joint with intersecting axes 
connected to a prismatic joint is proposed. In this case, 
safe joint actuation is implemented with wire-based 
mechanisms located outside the patient. Both joint 
designs are structurally optimized to achieve a 
maximum of inner space to incorporate further devices. 
For protection purposes, the module is covered with a 
bellow. Additionally, a stable manipulation platform is 
provided by employing a rigid endoscopic shaft and a 
connection to the patient’ s mouth gag. This also favors a 
reliable power and signal transmission. 

Low-level control is implemented on a main Controller 
which delegates position data to motor drivers. These 
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INTRODUCTION AND RELATED METHODS 

Feature matching is a core component in many 
endoscopic-imaging applications, such as Augmented 
Reality (AR) [1] or soft-tissue shape recovery [2]. 
Differently from feature tracking [3], feature matching 
recovers corresponding features between two images 
without any assumption about the small camera motion, 
or about the 3-D geometry of the observed scene (e.g., 
known organ motion due to periodic breathing). As 
such, they can be used for accurately recovering tracked 
features lost after strong organ deformation, fast camera 
motion, or prolonged occlusion. Existing feature- 
matching algorithms are characterized by limiting 
assumptions, and have not yet met the necessary levels 
of accuracy, especially when used to recover features in 
distorted or poorly-textured tissue areas. Figure 1 shows 
an example obtained by using the Hierarchical Multi- 
Affine (HMA) feature-matching algorithm [4].While 
HMA can predict well on average the position of 
corresponding features in the image after the sudden 
camera motion, HMA exhibits large errors (green lines) 
when predicting the position of features on the organ’ s 
boundaries (green circles). These large errors are caused 
by the absence of geometric constraints (the colored 
polygons correspond to affine mappings) because of the 
paucity of supporting image features in such highly- 
distorted or textureiess areas. In this paper, we present a 
novel feature-matching algorithm that accurately 
recovers the position of image features over the entire 
organ’ s surface. Our method is automatic, it does not 
require any explicit assumption about the organ’ s 3-D 
surface, and leverages Gaussian Process (GP) 
Regression [5] to incorporate noisy matches in a 
probabilistically sound way. We evaluated our approach 
over a large database of more than 100 in vivo 
endoscopic-image pairs, which showed an improved 
accuracy and robustness when compared to [6]. 

METHODS 

We refer to x.y) as a generic pair of corresponding 

features extracted in the image before, X 1 ', and after, X' 1 , 
the organ motion, respectively. 

The initial set of sparse matches from HN l {(x.y)}. 
are used to estimate a non-rigid and dense function,/ ( x) 
that maps any generic image point X*GX y to its 
corresponding image point y*£XX J ( x j can be written 
as,y*=/(x*)+€, where a 2 T) is a zero mean 

Gaussian noise with Standard deviation er. 

The sparse matches M are used as input to a Gaussian 
Process, which describes the posterior distribution of the 
latent function / as, p(f \ A4 ) oc p(M l/M/)’ where 



Fig.l The feature matching method in [4], exhibits large pixel 
errors (green line) when predicting the correspondences on 
organ’ s (kidney) boundaries. 

f(p) ^ G'P(f4 p), ^(p* q) .p and q being two generic 
vectors. Since a GP is completely determined by its 
mean and covariance matrix, we defme /dp) = p*as the 
deterministic mean of the non-rigid function / The GP 
covariance matrix is defmed as, k(p. q) - + 

er-£(| p- q ), where Ais the amplitude of the process 
variance, p is the length of the process Variation, and 
S (y)=l when y =1 or 0 otherwise. Note that l>( J M ) 
captures the interaction between the observed 
correspondences and the latent function f while p( f) 
represents the a-priori knowledge about the non-rigid 
mapping in terms of the GP mean and covariance. 

We denote as X and Y the 2*n matrices containing 
all the n 2-D HMA matcheA h . Also, consider the 2 
x m matrX*;: containing the m features to be 

precY fo, and Y as the corresponding points, to be 
predicted by the Gly J (x* , ) F e. Then, the 
joint distribution of both the observed values, Y, and of 
the target valu Y . under the prior is given by, 
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Where each (Lj) entry for the matrix K(X, X) 6 M n / 
K(X* : X) € iR m x n , and K(X ^ X, ) t are defmed 
as k{ x- { . } x j ) for vectors x and X ,belonging to X or X , , 
respectively. Then, the predicted values for the non- 
rigid function evaluated on X* are given by [5], 

Y* = X* + K(X.,X)K(X 1 X)‘ 1 (Y - X). 

The covariance matrix /■(*, ■) strongly depends on the 
values of the hyper-parameters A , / ) and er. For each 
image pair, we estimated these parameters by first 
computing the marginal log-likelihood of the model 
withrespect tosparse matches. Vf as, p(y|x. o. /;, A) — 
— 0.5(y — x) T K(Y, Y)^ 1 (y—x)— Ö.5 log jK(Y> Y)| — 
0.5 n log 2 tt. Finally, this log-likelihood is 
differentiated with respect to each hyper-parameter and 
maximized by using a conjugate-gradient optimization 
method. An example of these dense predicted (blue) 
points y* is illustrated in Fig. 2(a). 
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EXPERIMENTAL RESULTS AND DISCUSSION 

We validated our method on a large and manually 
annotated Surgical-Image dataset with more than 100 
cases extracted from more than six real partial- 
nephrectomy surgeries [4]. This dataset contains images 
with resolution 704x480 extracted before and after 
prolonged camera occlusions, organ deformation, 
camera retraction/insertion, and large illumination 
changes. We used our HMA Toolbox (implemented on 
Matlab and publicly available at [7]) to match SIFT 
features extracted from both images. The set of sparse 
correspondences - V t was selected as those SIFT matches 
with a reprojection error less than 15 pixels of error. We 
compared the accuracy of both our algorithm as well as 
of the HMA method, by measuring the pixel 
reprojection errors and the computational time on an 
Intel i7-2.20 GHz CPU with 8GB of RAM. 
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Fig. 2 (a) Estimated matches using the proposed approach. (b) 
The organ’ s boundaries are not mapped correctly with HMA. 
Our approach is more accurate. (c) Absolute pixel difference 
between the reference organ’ s image and the one mapped by 
HMA or by our proposed method. 

The results are summarized in Table I. The second 
column shows the average pixel-reprojection errors (in 
terms of mean and Standard deviation) for both 
algorithms. The third column contains the average over 
the 25% largest pixel-reprojection errors which reflect 
the mapping accuracy of the methods over the most 
challenging areas such as the organ’s borders or 
textureiess regions. 

Table 1 Average errors (pixels) over the 100 image pairs 


Mg. 

Avg.Err. |pi\els| 

Last Quartile Avg.Error 

Our method 

3.1313.09 

6.21 13.63 

HMA 

1.1 i 1.09 

s .60 k 4.23 


The proposed approach showed a reduction of the 
average error of 23%, with an increased computational 
cost of only 16.7% (15-pixels meshgrid), or of 60% for 
the dense matching. The average time required for 
Sparse feature-matching phase was 0.05±0.006s, while 
the GP regression phase requires on average additional 
0.011 ± 0.002s for the meshgrid, and 0.032 ± 0.002 
seconds for dense matching. 

The advantages of our proposed approach are 
evident in the qualitative example shown in Fig.2. The 


correspondences detected by our method between the 
two images are illustrated in Fig. 2(a). We only show 
the organ contour and the corresponding mesh to give 
an intuition of the mapping without cluttering the image 
with extra information. Fig. 2(b) shows the warped 
organ’s boundaries from the image before to the image 
after the occlusion for both our approach and HMA. 
Here the dotted lines indicate the boundaries of the 
warped organ, while the solid (blue) line represents an 
approximate manually selected ground-truth used as a 
reference for the comparison. Observe that our method 
can accurately map the organ’s boundaries. Finally, in 
Fig. 2(c) we show the absolute pixel difference between 
the warped patch and the image after the motion. It is 
evident that our method achieves reduced error (i.e., 
darker images). 

CONCLUSIONS 

We presented a feature-matching algorithm that 
accurately and densely recovers the position of image 
features over the entire organ’s surface. Our method is 
fully automatic, it does not require any explicit 
assumption about the organ’s 3-D surface, and it 
combines the speed of sparse feature-matching with the 
probabilistic modeling of GP Regression. We 
demonstrated the effectiveness of our algorithm under a 
large endoscopic dataset with 100 image pairs. The 
evaluation was performed by measuring the reprojection 
errors with respect to manually annotated ground-truth. 
The proposed algorithm achieved high accuracy when 
compared to state-of-the-art method, in particular for 
features closer to organ’s boundaries. In addition, the 
proposed method exhibits a comparable computational 
time with respect to sparse feature-matching methods. 
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INTRODUCTION 

While the use of preoperative imaging for minimally 
invasive surgical guidance has been successfully 
translated into the operating theatre [1], many of the 
difficulties associated with the reduced sensory 
experience remain. This approach works well as a 
roadmap, allowing the surgeon to appreciate the 
macroscopic anatomy (figure 1). However, it must be 
combined with an intraoperative phase designed to 
account for structural change induced by dissection, and 
tissue deformation in general. Adopting robotic partial 
nephrectomy as the exemplar intervention, this study 
assesses the feasibility of a novel technique for fusing 
dense Stereo scene reconstructions with intracorporeal 
freehand 3D ultrasound segmentations. In this instance, 
the fact that the resulting geometry can be viewed from 
any arbitrary position makes it a potentially invaluable 
aid in the pursuit of optimal nephron sparing resections. 

MATERIALS AND METHODS 

Following the work of Taylor et al. [2], porcine kidneys 
were injected with an agarose-based mixture to form 
completely endophytic phantom lesions. In practice, 
human kidneys will typically be covered in a layer of 
perinephric fat, further occluding the target. Currently 
practical intraoperative ultrasound methods [3] are 
suitable for verifying tumour location, but are very 
difficult to employ as a guide during resection. This is 
due to the fact that visualisation of the relationship 
between organ and lesion is transient, and that the 
viewpoint is necessarily constrained. 

The key observation is that it is possible to perform 
dense 3D scene reconstruction and freehand 3D 
ultrasound scanning in the same coordinate System. The 
captured data is therefore registered by construction, 
takes account of deformation, has permanence, and can 
be viewed from multiple virtual viewpoints, as required 
during resection. Moreover, since this can be performed 
quickly ‘on the fly’, it can be repeated as the dissection 
induces increasingly large-scale changes. 

Prior to data capture, the intrinsic and extrinsic 
Parameters of the da Vinci (Intuitive Surgical, 
Sunnyvale, CA, USA) stereo camera were calibrated 
[4,5]. Figure 1 illustrates the steps used to reconstruct 
surface geometry: image undistortion and rectification; 
generation of the disparity map using a semi-global 
block matching algorithm [6]; reprojection to 3D 
geometry and rotation back into the unrectified camera 
frame; construction of a pixel-dense triangulär mesh; 


decimation through quadric metric edge collapse [7]; 
and mesh smoothing using a windowed sinc(x) function 
interpolation kernel [8]. The resulting vertex positions 
were then projected, with camera distortion, into the 
current frame to give corresponding texture Coordinates. 



Fig. 1 Surface reconstruction (left-to-right from top): live 
human intraoperative guidance; porcine kidney with lesions; 
disparity map; and surface geometry following decimation and 
smoothing. 

Tracked in real time with the help of a robust vision- 
based algorithm [3], an Aloka UST-533 microsurgery 
probe (Hitachi Aloka Medical Ltd., Tokyo, Japan) was 
first calibrated using the dual pattern approach, as 
shown in figure 2. Ultrasound slice images and their 
respective transformations into the camera coordinate 
System were accumulated, such that a complete volume 
could be extruded using trilinear interpolation. The 3D 
geometry was then segmented [9] and realised as a 
mesh, or visualised directly with HDVR Volumetrie 
rendering [10]. 



Fig. 2 Ultrasound image acquisition (left-to-right): probe 
calibration using water-filled Z-phantom and dual pattern 
technique; and typical ultrasound slice showing subsurface 
extent of phantom tumour. 
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RESULTS 

Figure 3 illustrates the resulting surface and subsurface 
reconstructions, viewed from different positions, 
together with a cross-section of the porcine kidney for 
reference. In addition to straightforward alpha-blending, 
an extension of the inverse realism [11] technique was 
used to combine the reconstructions. Rather than use a 
fixed window in the viewport, the new two-pass method 
first renders a mask, painted onto the surface mesh 
texture, to a temporary frame buffer. This is then 
combined with a rendering of the virtual organ surface, 
before being composited with the 3D ultrasound 
representation. This way, the window appears to move 
with the organ surface and reveals subsurface anatomy 
as the viewpoint is rotated and translated. In practice, 
one or more such views will be visualised in Stereo via 
the console TilePro™ auxiliary video inputs. 



Fig.3 Reconstruction results (left-to-right from top): partially- 
truncated tumour Segmentation; alpha-blended subsurface 
anatomy rendered both from its mesh representation and 
volumetrically with a false colour transfer function; alternative 
views showing the inverse realism extension window; and an 
actual tissue cross-section for comparison. 

DISCUSSION 

The study results acquired to date strongly suggest that 
the proposed multimodal reconstruction method will 
prove to be an invaluable image guidance tool. 
Furthermore, since all of the physical components are 
ready for human use, in terms of Sterilisation, for 
example, then once validated, the approach can 
immediately translate to the operating theatre. 

Quantitative validation experiments are planned, in 
which 3D CT scans of contrast-enhanced lesions will be 
compared against reconstructed geometry. Although the 
reported accuracy [3] of the probe calibration process is 


high, with mean ultrasound and direct sight super- 
imposition error less than 0.7mm, it is necessary to 
prove that any deformation caused by the sweeping 
back and forth of the probe is not significant. Further 
developments to mitigate such an effect are also 
imminent. These include autonomous robot control of 
probe sweeps, based on reconstructed surface geometry, 
and the introduction of force sensors at the probe- 
manipulator coupling. 

Opportunities also exist to improve freehand 
ultrasound slice interpolation and Segmentation [12]. In 
addition, disparity map averaging over multiple video 
frames will be assessed as a means of improving the 
accuracy of dense surface reconstruction. Clearly, the 
described approach lends itself well to other surgical 
interventions, including non-robotic procedures, and can 
act as an effective canvas for the persistence and 
subsequent review of pathological data acquired at the 
cellular level, using microscopic imaging modalities. 
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INTRODUCTION 

One on the main benefits of introducing robots in the 
operative room is to increase the precision and accuracy 
of the surgical tasks, which follows a given clinical 
workflow, i.e., the knowledge from the surgeons. So, it 
is convenient to have proper know-ledge sharing and 
communication between surgeons and robots. 

In this paper, ontologies are used to model the 
workflow/knowledge required to accomplish each 
surgery task in detail, allowing successful surgeries, and 
also to represent that knowledge in a machine-readable 
format. The ontological model obtained, i.e., its formal 
description, is of major interest to Setup a surgical robot, 
e.g., a Service robot manipulator applied in surgery, that 
will perform the surgery in a co-worker scenario with 
the surgeon, delivering the expected increase in 
accuracy and precision. The application presented in 
this work was developed under the HIPROB project 
http://www. echord. info/wikis/website/hiprob , and is 
related to Hip Resurfacing (HR) prosthesis surgery. It is 
an alternative to Total Hip Replacement (THR), 
especially for younger patients that may require hip 
joint revisions in the future. However, HR is more 
complex than THR, and requires precise and accurate 
following of the planned surgical workflow. To the 
authors’ best knowledge, there is no generic framework 
for process modelling and formal definition that is 
specific to robot-assisted surgery. Moreover, in this 
work is included an ultrasound probe to track the bone 
alignment during surgery, as presented in [1]. 


[4], the Open Robots Ontology ORO [5] and KnowRob 
[6]. They were considered as the basic source of 
contents and partially covered the knowledge needed to 
develop the proposed ontology. The ontology was 
implemented using Protege, a free and open-source 
platform that provides a suite of tools to create domain 
model and knowledge base application using ontologies. 
The underlying language for ontology definition used 
by Protege is the Web Ontology Language, OWL. 


RESULTS 


The classes and their properties were identified using 
the existing defmitions on the base ontologies, like 
BodyStructure, Surgeon from the BioOntology [3] and 
ImagingDevice, SurgicalDevice defined in [4]. From 
this and the functional requirements obtained from the 
surgical workflow for HR surgery, new classes and 
properties were defmed for the pre and intra operative 
surgery phases, e.g., IntraOperativePhase and 
PreOperativePhase. Also ActuatedMechanism was 
defmed and its sub-classes (derived from ISO Standard 
no.8373 and [7]). A particular property from the 
proposed ontology is that the ultrasound probe 
US _probe (a sub-class of USjmaging ) isAttachedTo the 
EndEffector and isUsedln the IntraOperativePhase. 
Computer tomography imaging CT_imaging isUsedln 
the PreOperativePhase. Please refer to figure 1 to 3, 
where portions of the ontology are depicted. 
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MATERIALS AND METHODS 

While developing the orthopaedic surgery ontology, for 
HR, a five-step development process was adopted, 
called ontology development 101 [2]. Basically, this 
process defmes the domain and scope of the ontology; 
reviews existing ontologies; identifies classes and 
properties; identifies and implements an upper ontology; 
and implements the ontology in a formal representation. 
The ontology domain and scope was defmed when the 
author was asked to automate the HR surgery, following 
the contacts with surgeons in Lisbon and Castelo 
Branco Hospitals. A set of interviews was conducted 
with the surgeons to better define the domain of the 
approach. Existing ontologies from the medical and the 
robotics fields were reviewed, e.g., from the BioPortal 
[3], OpenCyc ( http://www.cvc.com/platform/opencyc ), 
the Surgical Ontologies for Computer Assisted Surgery 
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Fig. 1 A fraction of the orthopaedic ontology, related to the 
robot and medical devices. Solid lines indicate dass hierarchy, 
and dashed lines connections between classes. 

The next Step is to implement an upper ontology. For 
that the OpenCyc was used because it is interchangeable 
amongst other robotic and biomedical ontologies. Also 
provides a basic structure, generalizable to domain 
specific ontologies. OpenCyc is used as the main source 
to design the structure and also to define HR surgery 
ontology classes and its properties. 


The Hamlyn Symposium on Medical Robotics (2013) 


61 



' •, M . J. -1-iJjjOC* 


' * l'rtaginqÖuvioG 



. '■* trYtf&OpmwF 1 
lau 






1 

4 fan*LnnQ)Fnr™v 




‘ 4 FwiiUPn: 

+ 0 U9 irnrnjn^ j 




| 0 Cjtftf Sl TriUtvat | 


US /TWU 



' (» Suivnfei 


k EdlMeTrixF.ir 


| ' # Ct .MMatW 


+ 0 P-vC^kh™»*^ 


Fig. 2 A fraction of the orthopaedic ontology, related to 
imaging concepts. Solid lines indicate dass hierarchy, and 
dashed lines connections between classes. 


The ontology was implemented in OWL, and a set of 
classes was defined specifically for the HR surgery 
Setup, e.g., BoneDrill , USProbe. The knowledge ma- 
nagement module for cognitive architectures in robotics 
ORO, is used to map the robot Services to the ortho- 
paedic ontology. ORO is an open-source framework that 
can be used within the ROS framework to control a 
robot. This model includes, amongst other features, the 
capability to knowledge Classification based on SWRL 
rules, using Pellet, an OWL 2 Reasoner for Java 
http://clarkparsia.com/pellet . 

The ORO Server performs reasoning as the following 
example: 

for tools in kb["?tool rdf:type SurgicalDevice ", 
"?tool isAt EntireHeadOfFemur"\. 

print tools. 

this query can retum the Bone_drill or an 
Optical_Tracker , i.e., two tools that can work in the 
head of the femur. 
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Fig. 3 A fraction of the orthopaedic ontology, related to 
Spatial and Mathematical things. 

DISCUSSION 

For HR surgery several medical imaging Systems could 
be useful like magnetic resonance, Computer 
tomography, and ultrasound. In the current application, 
two types are needed, ultrasound for intra-operative 
Operation and Computer tomography for the pre- 
operative stage. Surgical tools, some to be used by the 
robot, e.g., Borte Drill, and other to be used by surgeons 
like reamers, should be defined. Knowledge from the 
human body must also be shared with the robot via the 
ontology, e.g., femoral head. HR robotic surgery 
specific equipment like a Service robot, and the optical 


tracking Systems, should be set in the ontology. In the 
following, are discussed two sub-sets of knowledge 
representations for the tackled surgery. 

For the robotic drilling of the head of the femur, 
figure 1 presents the related classes. The Robot, 
Surgical Devices and Imaging Devices all derive from 
the dass Material Object. It is possible to verify that the 
surgeon can also operate the Bone Drill , e.g., in a co- 
worker scenario. Usually, a second surgeon operates the 
US Probe. A second robot, depending on the chosen 
setup for HR Robotic Surgery, can also operate it. 

Figure 2 depicts the medical imaging devices needed 
for HR Robotic Surgery in a minimal setup [1] i.e., 
CT_imaging (pre-operative scenario), US Jmaging and 
Optical Tr acker (both in the intra-operative scenario). 
The CT_imaging device performs a 3DScan of the 
entire femur, represented as a PointCloud , used for 
intra-operative registration with the 3D data obtained 
from the Polaris Tr acker and the US _probe , when the 
latter obtains images from the shaft of femur. 

By this way, in this paper, is shown that is possible 
to represent knowledge, using ontologies, for Surgery. 
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INTRODUCTION 

Critical to robotic thyroidectomy is the deployment of a 
custom designed lifting device inserted through an 
axillary incision. Several bed-mounted retractors have 
been developed for clinical practice. Compared to South 
Korea, where over 3000 procedures have been 
performed, patients in the UK and US undergoing 
robotic thyroidectomy have significantly larger thyroid 
pathology. Their BMI is also significantly higher 1 . 
These factors necessitate a larger working space than 
may be afforded by the existing devices. Improving 
exposure of the surgical field is essential for the 
technique to become popularised in a Western 
population. 

We have previously reported an alternative retractor 
design . This is patient-mounted rather than bed- 
mounted to increase the working space. The initial 
prototype underwent load testing and two sequential 
design iterations following cadaver evaluation. The 
prototype tested here is modular, creates a 5 cm vertical 
opening and is based on a secure interaction with the 
clavicle (Fig 1). It has a smaller external footprint and 
locking mechanism, smoothed edges and reshaped blade 
in comparison with the previously reported iteration . 


The objective of this study was to compare this latest 
prototype (the Imperial retractor) with 2 bed-mounted 
devices used in clinical practice (the Chung and Modena 
retractors) (Fig. 2). 



Fig. 1 : The Imperial retractor. 


MATERIALS AND METHODS 

Robotic thyroidectomy was performed using the 
daVinci SI robot [Intuitive Surgical] in a fresh frozen 
cadaver by three thyroid surgeons familiär with the 
robotic approach. Initial exposure was achieved through 
a 6cm axillary incision with the cadaver positioned in 
Standard fashion (supine with ipsilateral arm elevation). 
Using a Chung retractor coupled with a spring balance, 


the load required to create a 5cm vertical opening was 
measured. The time taken to assemble, insert and secure 
the three retractors was recorded. The robot was then 
docked with four instrument arms (5mm Debakey 
forceps, 5mm Maryland dissector, 8mm Prograsp and 
12mm dual channel endoscope) inserted through the 
incision. 

The console surgeon performed a series of standardized 
tasks that reciprocated key steps in robotic 
thyroidectomy: i) mobilization of superior thyroid pole 
ii) dissection off the recurrent laryngeal nerve (RLN) iii) 
mobilization of the inferior pole iv) dissection of 
isthmus and contralateral lobe. An interactive task was 
also performed using a pledget exchanged between the 
console and bedside surgeons. Three surgeons each 
performed the console and bedside tasks. 



Fig. 2 From left to right, the three retractors used in this study: 
Modena retractor, Imperial retractor and Chung retractor. 

Evaluation of the retractors and ease performing the 
tasks was recorded on a 1-10 categorical Likert-type 
scale (LTS) with 10 being the most positive outcome. 
The number of instrument arm clashes and repositions 
that occurred while performing the tasks was recorded. 
Retractor stability was assessed by measurement of 
slippage out of the body. Finally, the ipsilateral arm was 
dropped to the bedside and observations of the exposure 
of the thyroid in this configuration were recorded. 

RESULTS 

The load of 20-25N to create access was considerably 
lower than the previously reported 84N and is attributed 
to the use of a ‘fresh- frozen’ cadaver rather than a 
preserved specimen with phenol and glycerine. 

Results relating to the ease with which the three 
retractors could be deployed are shown in Table 1. 
Surgeon familiarity with the bed-mounted designs and a 
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crude fastening device on the Imperial retractor account 
for the lower satisfaction rating for the ease of setup. All 
retractors suffered from tissue slippage, but exposure 
was not adversely affected with the Imperial design. 


Table 1 Deployment of the retractors. A low trauma LTS 
score is desirable, while a high LTS score indicates 
satisfaction with the ease of setup and tissue slippage. 


Task 

Chung 

Modena 

Imperial 

Setup time (s) 

113 

208 

60 

Ease of setup (LTS) 

8 

8 

7.7 

Tissue slippage (mm) 

12 

25 

12 

Tissue slippage (LTS) 

6.0 

1.0 

10.0 

Trauma (LTS) 

10.0 

10.0 

1.0 


All retractors enabled the console surgeon to perform 
the tasks for an ipsilateral lobectomy. However, the 
exposure created by the Imperial retractor (Fig. 3) 
resulted in higher LTS feasibility scores (Table 2). The 
prevention of tissue encroachment enabled more 
flexibility in positioning the robot arms hence fewer 
cumulative clashes. This was particularly evident when 
excising the contralateral thyroid lobe. 


Table 2 Console surgeon evaluation of tasks with mean LTS 
score, Standard deviation and (cumulative number of clashes). 
A high LTS score is desirable. 


Task 

Chung 

Modena 

Imperial 

Superior pole 

6.7±1.2 (6) 

8.0±1.0 (6) 

9.3±0.6 (0) 

RLN 

7.7±0.6 (0) 

8.0±0.0 (0) 

8.7±1.2 (0) 

Inferior pole 

6.7±0.6 (3) 

7.0±2.0 (4) 

8.3±0.6 (1) 

Contralateral lobe 

1.0±1.7 (6) 

5.7±1.2 (7) 

7.7±0.6 (0) 

Pledget exchange 

7.7±1.2 (3) 

7.0±1.0 (9) 

9.3±0.6 (0) 



Fig. 3 The exposure offered by the Imperial retractor. 


A well-maintained working volume and flexibility in 
the docking position of the robotic arms (Fig. 4) 
accounted for the bedside surgeons’ higher mean 
feasibility scores with the Imperial retractor compared 
to the bed-mounted alternatives (Table 3). 


Table 3 Bedside surgeon evaluation of tasks indicated by 
mean LTS score, Standard deviation and (cumulative number 
of repositions). A high LTS score is desirable. 


Task 

Chung 

Modena 

Imperial 

Superior Pole 

4.0±3.0 (2) 

6.7±1.5 (5) 

9.3±1.2 (0) 

RLN 

4.7±2.1 (0) 

6.0±1.7 (0) 

9.3±0.6 (0) 

Inferior Pole 

3.7±2.9 (1) 

5.3±2.1 (9) 

8.7±1.5 (0) 

Contralateral Pole 

1.0±1.0 (2) 

3.3±3.2 (6) 

7.3±1.2 (0) 

Pledget exchange 

4.7±2.5 (0) 

6.0±1.7 (0) 

9.3±0.6 (0) 


DISCUSSION 

In stability and deployment the Imperial retractor is 
favorable to the existing bed-mounted designs. By 
preventing tissue encroachment, the Imperial device 
retains exposure despite tissue slippage. The Imperial 
retractor also remained stably fixed on the clavicle. In 
contrast, the Modena retractor required complete 
repositioning due to tissue slippage. Elimination of 
clashes and repositions, with the ability to perform total 
thyroidectomy, were the main benefits of the Imperial 
device. No retractor allowed sufficient exposure when 
the ipsilateral arm was dropped to the bedside 



Fig. 4 External view of the Imperial retractor and docked 
daVinci robot arms during console surgery. 

A further iteration based on the outcome of the robotic 
trials is shown in Fig. 5. Access to the incision is 
opened up by creating a single central support around 
which all the robotic arms can easily be directed. A 
ratchet mechanism is employed for improved 
deployment and variable opening heights. 
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Fig. 5 Rapid prototype of retractor with arms detached, based 
on outcome of the robotic trials. Revised features highlighted. 

In conclusion, the Imperial retractor is a flexible and 
easily deployed alternative to existing retractors. 
Enhancements based on the robotic testing described 
here will lead to imminent clinical trials. 
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INTRODUCTION 

Minimally Invasive Surgery (MIS) driven by patient 
demand and benefits in outcome, imposes certain 
limitations on the surgeon. Sensorimotor discordances, 
distorts the surgeon’ s perception making it mentally 
more challenging to perform MIS than its traditional 
open counterpart, thereby requiring longer to attain 
proficiency [1]. In particular, MIS knot-tying (MISKT) 
is arguably the most challenging surgical manoeuvre 
and serves as a barometer to perform advanced MIS. 
Moreover, the influx of new technology that Supports 
MIS necessitates the operators’ vigilance to attend to 
auditory alarms that alerts the surgeon to a faulty 
technical device or declining Status of the monitored 
patient. These may also be in the form of interruptions 
from other members of the team that require immediate 
decisions on patient management outside of the 
operating room. On average, surgeons are interrupted 
13.5 times whilst operating [2]. Mental resources are 
finite and the impact of the secondary task (e.g. decision 
making) on the simultaneously performed primary task 
(i.e. technical) has shown that with expertise there is 
variable degradation of one or both task performances 
[3, 4, 5]. We hypothesize that with increasing task 
complexity (MIS versus open knot-tying) and in the 
presence of a secondary task (sustained auditory 
attention) an increase in cognitive load and performance 
degradation would be demonstrable. Unlike other 
studies that measured performance degradation and 
mental workload using subjective questionnaires, stress 
and eye-gaze patterns, we aim to use 
electroencephalography (EEG) to directly observe these 
phenomena. 

MATERIALS AND METHODS 

Participants (n=9) comprised of 8 residents (PGY3-8) 
and 1 attendee aged 32.2 ±3.3 years (mean ± S.D.). 15 
minutes for warm-up to practice surgical manoeuvres 
relevant to the trial were mandatorily provided. 
Participants initially performed open hand knot-tying 
(OHKT) and MISKT (primary tasks) as illustrated in 
Figure 1 , for a fixed period of 200 seconds (Application 
Data). Subsequently, in addition to performing 
OHKT/MISKT participants were required to 
simultaneously count the number of high-toned beeps 
randomly introduced amongst a series of low tone beeps 
(secondary task) for same time period (Calibration 
Data). The order of tasks was randomized to minimise 


learning and other temporal effects, e.g. fatigue. Düring 
control tasks participants performed the secondary task 
(auditory) alone. EEG was recorded from a 32 
impedance-optimized electrodes (Brain Products, 
Gilching, Germany) transmitting signals wirelessly to a 
Standard personal Computer. A passive Brain Computer 
Interface (BCI) [6] was setup using the toolbox 
BCILAB [7] to discriminate trials from the low 
cognitive load condition (OHKT) to those from the high 
load condition (MISKT). Features were extracted from 
logarithmic band-poer estimates of the signal with a 
spatio-temporal filter derived by the spectrally weighted 
Common Spatial Patterns method [8]. Three filter pairs 
were calculated with optimized frequency weights 
between 7 and 23 Hz. A regularized linear discriminant 
analysis (LDA) was used to estimate an optimal 
discriminating hyper-plane in the resulting feature 
space. Performance estimates were calculated on the 
Calibration Set by a five times repeated five-fold cross 
validation with margins of five [7]. The resulting 
Classification approach was then applied on consecutive 
epochs of one-second length from Application Set to 
simulate an online application. Secondary performance 
measures for the primary task were number of 
OHKTs/MISKTs performed in the time period and for 
the secondary task were accuracy in the number of high 
toned beeps counted. Subjective workload (NASA- 
TLX) was assessed based on task complexity [9]. 



Fig. 1 Participant performing open hand knot-tying (left) and 
intracorporeal laparoscopic knot-tying (right) in a box trainer, 
whilst brain behaviour was assessed with EEG. 

RESULTS 

Participants found MISKT significantly harder than 
OHKT (Wilcoxon signed-rank test, (p=0.012) and the 
secondary task alone (p=0.025) as indexed from group 
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averaged NASA-TLX scores displayed in Table 1. 
There was no significant difference between NASA- 
TLX scores for OHKT and the auditory task (p=0.55). 
Participants found dual tasks significantly harder than 
mono tasking. 


Table 1 Group averaged NASA-TLX scores for each task. 


Task 

NASA-TLX score 
(mean ± S.E.) 

p value 

Mono OHKT 

16.99 ±2.85 

Dual vs. Mono 
OHKT (0.018) 

Mono 

MISKT 

48.79 ±6.07 

Dual vs. Mono 
MISKT (0.012) 

Auditory 

24.33 ±6.07 

Auditory vs. Dual 
OHKT and MISKT 

(0.017 and 0.011) 

Dual OHKT 

51.03 ±7.42 

Dual MISKT 

67.55 ±6.91 


Although performance deterioration was observed in the 
secondary task with increasing primary task complexity 
it did not reach Statistical significance. Average number 
of OHKTs and MISKTs performed was 117.7 and 13.1 
respectively. EEG data of participant 1 and 2 were 
insufficient and for participant 8 was disregarded as it 
lacked markers for each task. An example of subjects 
activation and frequency weights calculated by 
Common Spatial Patterns (Figure 2) displays mental 
workload, bilateral motor, auditory and visual activity 
as processes underlying the Classification. 
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Fig.2 Activation patterns and frequency weights calculated by 
Common Spatial Patterns. 


Average Classification of subjects into low and high 
cognitive demand (OHKT vs. MISKT) was 89.4% on 
calibration and 87.8% on application. One subject 
(subject 5) showed comparatively poorer Classification 
compared to rest who accurately classified up to 100% 
as seen in Table 2. On a subject level, successful 
Classification of work into high and low cognitive 
demand conditions for each second of the experiment 
was achievable. The crossvalidation estimates are 
accurate, showing no significant difference (p<0.01) 
from the application performance. 


DISCUSSION 

Mental workload varies depending on the operative task 
complexity. Düring high mental workload States that 
demand greater attentional resources they are unlikely 
to cope with distractions due to a lack of free cognitive 


resources. The results from Classification of EEG data 
alone of 1 second length demonstrate accurate and 
reliable discrimination of a low-demand (OHKT) and 
high-demand task (MISKT). This work highlights the 
potential to build a platform that detects online the 
cognitive load Status of the surgeon and visually 
communicates that information to the rest of the 
operative team by employing a passive BCI System. An 
example of the applicability of such a System would be 
to help inform other members to avoid any interruptions 
when the surgeon is in a high cognitive load state. 
Future work needs to investigate how well the current 
methodology translates to low and high demand 
conditions in vivo. 


Table 2 Classification accuracies for each subject from 
crossvalidation and application of the classifier. 


Participant ID 

Calibration (%) 

Application (%) 

1 

87 

n/a 

2 

92 

n/a 

3 

84 

89 

4 

91 

76 

5 

67 

70 

6 

97 

94 

7 

98 

100 

8 

n/a 

n/a 

9 

99 

98 

Average 

89.4 

87.8 
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INTRODUCTION 

One of the most promising approaches to recently 
emerge for the active guidance of miniature intra- 
corporeal robots (such as capsule endoscopes, involves 
external magnetic fields interacting with a small 
permanent magnet inside the device [1, 2]. Depending 
on the proposed method, the applied force/torque 
vectors are adjusted by either Controlling the current 
supplied to stationary electromagnetic coils, or by 
changing the position/orientation of one or more 
permanent magnets outside the patient. In this context, 
reducing the frictional forces encountered by the micro- 
robot would ensuingly lower the required strength of the 
external magnetic fields, yielding multiple benefits for 
the design and functionality of the Overall System, 
related to cost, size, safety, and EMI considerations, 
while potentially also allowing for smoother and more 
precise steering of the device by the doctor. Although 
various studies have investigated the reduction in 
friction that occurs when vibrations are superimposed to 
the contact surface between metals and/or plastics [3], 
few have considered this concept for intracorporeal 
medical devices [4, 5]. 

MATERIALS AND METHODS 

Computational Model: A platform of mass M moves 
over a Substrate with normal-wise compliance, modeled 
as a Voigt element with damping b and stiffness k (Fig. 
1). An eccentric mass m , rotating with constant velocity 
co at a radius r about the platform ’s center-of-mass (9, 

generates normal and tangential harmonic excitations, 

2 

respectively obtained as C y (t) = mrco cos (cot) and C x (t) = 
mrco sin(ft>0. The platform’ s equations of motion are: 

Afy(t) = Mg + C y (t)~ F N (t) (1) 

Mx(t ) = F exl (t) +C x (t)~ F R (t) , (2) 

where F N ( t ) = ky{t) + by(t) is the normal contact force 

(applied fory(f) > 0), and g is the acceleration of gravity. 
The horizontally-applied external force Fext represents 
the effect of the magnetic guidance System, and is here 
assumed to be constant (in practice it would depend on 
the position /orientation of the platform with respect to 
the external magnetic field). F R (t ) denotes the platform- 
substrate friction force, here described by linear viscous 
damping combined with a modified Version of the Dahl 
model, the hysteresis characteristic of which has been 
shown to be significant in the context of frictional 
reduction by vibrations [3]. Specifically, 



Fig. 1 Mechanical model of the System. 
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where 8 is the viscous coefficient, ju(t) is the time- 
varying Dahl friction coefficient, pk is the kinetic 
Coulomb coefficient, x b is the “breakaway” displace- 
ment, and a determines the stress-strain curve shape [3]. 

Ex-vivo experimental setup: Fig. 2 depicts the setup 
for assessing the effect of vibrations for movement over 
ex-vivo bovine liver. The test platform, made from 
ABSplus, has a 39 mm x 18 mm footprint, and 
integrates a 010 mm pager motor (nominal velocity 
1 1200 rpm @3 V) for generating the vibrations. A high- 
precision digital force gauge ( Alluris FMF210A5 ), 
mounted on a servo-controlled linear stage, measures 
the frictional resistance of the prototype, as it is being 
pulled over or through the environment of choice, at 
different imposed velocities. The laser interferometer 
(micro-Epsilon optoNCDTl 402-10) allows measure- 
ment of the capsule ’s vertical displacements, resulting 
from the vibrations, with a 1 pm resolution. Data from 
the two sensors are streamed in real-time to a Work- 
station PC, where a LabView-based user interface 
facilitates adjustment of the various motion control and 
data acquisition parameters. 



Fig. 2 Experimental setup employed in the ex-vivo tests. 


The System parameters, corresponding to the model (1)- 
(3), are shown in Table 1. The values for k , b , //^and 3 
were obtained by appropriate identification tests, while 
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Table 1 Simulation parameters. 


M: 9.96- 10“ 3 kg 

k : 181.2 Nm“ 1 

jup 0.89 

m\ 0.32 • 10 -3 kg 

b: 1.29 N sm" 1 

a : 0.5 

r: 0.86 • 10 -3 m 

ö : 3.94N m -1 

xp. 1 • IO -3 m 


those for a and x h were estimated on the basis of 
matching the simulations with the experimental data in 
Fig. 4. 

In-vivo experimental setup: A vibratory capsule 
prototype, powered by an on-board battery and 
weighing 5.8 g, was connected via surgical string to a 
force sensor mounted at the distal link of a robot arm 
(Fig. 3). Using an endoscope, the capsule was inserted 
in the anaesthetized swine ’s colon, about 350 mm from 
the anus. The robot arm was then commanded to move 
so that its distal link translated horizontally at a constant 
speed of 6 mm/s, with the sensor collecting data as the 
capsule was being pulled out of the colon. Two such 
runs were performed, one without vibrations and one 
with the vibratory motor operating at 3.7 V. 



Fig. 3 Experimental setup of the in-vivo tests. 


RESULTS 

The ex-vivo test- stand was used to measure the force 
required for pulling the platform at a constant velocity 
of 6.13 mm/s over the liver sample, in the presence of 
vibrations, whose frequency was adjusted via the 
voltage applied to the pager motor. Corresponding 
simulations of the model (1) — (3) were performed, 
where, for different values of co, the magnitude of F ext 
was iteratively updated to obtain its appropriate value 
for the System to achieve an average steady- state 
velocity of 6.13 mm/s. The combined results, shown in 
Fig. 4, indicate that the use of vibrations yields 
considerable frictional reduction, which increases (up to 
41%, when co = 220 Hz) with the Vibration frequency. 


Average friction force (traction velocity: 6.13mm/sec) 



Vibration frequency (Hz) 


Fig. 4 Average friction, as a function of the Vibration 
frequency, for movement of the vibratory platform over 
bovine liver. 


Moreover, the Simulation results can be seen to closely 
match the experimentally obtained ones. 

The in-vivo results, shown in Fig. 5, indicate a 39% 
reduction in the average force when vibrations are 
employed during movement through the porcine colon, 
noting that forces considerably higher than those for 
movement over liver were recorded, possibly due to the 
increased pressure exerted on the capsule from the walls 
of the colon. 



Fig. 5 Forces recorded during the in-vivo tests. 

DISCUSSION 

Our fmdings suggest that vibrations can effectively 
reduce the average friction forces for magnetically- 
guided intracorporeal devices. Concems regarding the 
vibratory motor power consumption could be alleviated 
by its intermittent, on-demand activation, and/or by the 
adoption of wireless power transfer technologies [6]. 
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INTRODUCTION 

Visualisation of oxygenation of the vasa vasorum as a 
means of assessing oxygen transport from the lumen to 
the vessel walls may be a valuable tool for the study of 
haemodynamics at anastomotic junctions [1]. This 
information could then be used for prediction of 
neointimal hyperplasia (NIH), a thickening of vessel 
walls, which itself is a cause of anastomotic failure [2]. 
Currently, in the case of fistula failure for example, NIH 
is not specifically detected in individual patients but has 
been identified in studies as a cause of Stenosis [3]. 
Intraoperative measurement of vessel wall oxygenation 
would allow further investigation of the correlation 
between oxygenation and clinical outcomes. 

Recent work by our group has shown the 
applicability of multispectral imaging in the 
measurement of perfusion changes intra-operatively for 
robotic surgery applications in the bowel [4], and for 
organ viability monitoring following anastomoses of 
transplanted organs [5]. These spectral imaging Systems 
used liquid crystal tuneable filters (LCTFs) with 
convenient electronic Switching capabilities to extract 
high spectral resolution information. However, LCTFs 
have transmission values of less than 50%, which must 
be compensated for with long integration times [4]. 

In this paper, a laparoscopic adaptation of a fast 
Filter wheel multispectral imaging (MSI) System is 
demonstrated using the recently-developed SpectroCam 
platform (Ocean Optics, Ine., USA). This allows high- 
throughput imaging at multiple spectral bands at 30 Hz. 
Initial in vivo results obtained using the System in a 
porcine subject show changes in oxyhaemoglobin at the 
anastomosis site during arterio-venous fistula creation. 

MATERIALS AND METHODS 

The laparoscopic System described in this paper is built 
on the SpectroCam platform (Fig. 1), which is a fast 
filter wheel multispectral imaging device capable of 
accommodating up to eight separate Filters. The spectral 
properties of the current configuration are shown in Fig. 
1 (a). Acquisition of five multispectral Stacks, of eight 
high-resolution (1392x1040 pixels) images each, can be 


accomplished in 0.27 s. A 30° laparoscope (KARL 
STORZ GmbH & Co. KG, Germany) was attached to 
the System using a custom-made F-mount adapter. This 
also housed a 50 mm focal length imaging lens that 
fixed the working distance at approximately 3 cm. 


Fig. 1 (a) Transmission characteristics of Filters used, covering 
the visible ränge, (b) Photograph of MSI System with 
laparoscope and xenon white light source attached. 

The System was used to monitor perfusion at the 
anastomosis site during the creation of an arterio-venous 
fistula in a 75 kg Yorkshire pig (conducted under a 
Home Office licence). The carotid artery and jugular 
vein were exposed through a 12 cm incision in the neck 
and then joined in an end-to-side anastomosis. In order 
to complete this, the artery was clamped upstream of the 
anastomosis site, flushed with saline and clamped again 
downstream. The jugular was clamped and divided in 
preparation. Static MSI images of the artery in its native 
state, and after clamping and flushing were acquired. A 
time series was then acquired as the clamps were 
released to monitor the perfusion change in the vein. 

Perfusion and oxygenation changes were assessed 
by measuring the absorbance of light at each Filter 
waveband. Reflected light was compared to the 
reflected light intensity from a reference Standard at 
each waveband to calculate the experimental 
absorbance, A (Eq. 1). 

A = -1n(// 1 R ) (1) 

where / is the measured intensity at a particular 
waveband and I R is the corresponding intensity detected 
from a reflectance Standard. Assuming that absorption 
by haemoglobin is the dominant mechanism of light 
attenuation and that Scattering losses are flat across the 
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wavelength ränge of interest then a simple model [6] 
can be used to predict^T, as shown in Eq. 2. 

A — Hbo ^ \Hb]ßHb ^ (^) 

where [ Hbö 2 ] and [Hb] are fractional concentrations of 
oxy- and deoxy-haemoglobin respectively, s is the 
known extinction coefficient of haemoglobin 
(convolved with the Spectrum of each filter in Fig. 1) 
and a is a constant term accounting for Scattering 
losses. Experimentally measured absorbance spectra 
from each pixel location in the MSI stack were then fit 
by the three parameter model in Eq. 2 to calculate 
[Hb0 2 ] and [Hb] at each point in the field-of-view. 

RESULTS 

Figure 2 shows the dissected porcine aorta prior to 
anastomosis isolated with a sterile plastic sling. The 
colour image shown in Fig. 2 (a) was assembled from 
three of the wavebands centred on 475, 575 and 650 nm. 
In the oxyhaemoglobin colour map (Fig. 2 (b)), the fine 
network of the vasa vasorum is visible in the central 
portion of the image. Superficial droplets of blood 
resulting from the dissection obscure the extreme ends. 



Fig. 2 MSI of the porcine aorta. (a) Colour RGB. (b) Map of 
oxyhaemoglobin concentration. 




Fig. 3 Arterio-venous anastomosis. The jugular vein is lying 
directly on top of the aorta (white arrows) and the location of 
the sutures is indicated (white circle). The change in 
oxyhaemoglobin concentration over time following the release 
of the upstream aortic clamp is shown at (a) t = 0 s (b) t = 72 s 
(c) t = 114 s. Inset: side view of the anastomosis. (d) [Hb0 2 ] 
time course for region of interest indicated in (a). 

Following the clamping and flushing procedure 
described earlier, the jugular vein was joined to the 
aorta in an end-to-side anastomosis. Figure 3 (a) shows 
the site immediately prior to the release of the clamps 
with the vein lying directly on top of the artery. 


The high-speed acquisition of the MIS System 
allows capture of the haemodynamics as highly 
oxygenated arterial blood begins to flow into the vein. 
The increase in oxyhaemoglobin is visible in Fig. 3 (d) 
along with the characteristic dilatation of the vein due to 
the increased arterial pressure. 

DISCUSSION 

A new laparoscopic implementation of a multispectral 
imaging System has been presented. The fast filter 
wheel allows rapid detection of multispectral image 
Stacks, minimising the influence of motion artefact. This 
was seen in the in vivo data in the capture of the venous 
dilatation. The transmission filters used have high light 
transmission (almost 100%), exceeding alternative 
Systems such as FCTFs and resulting in reduced noise. 

The results described here combine absorption by 
blood in the lumen of the vessel as well as the vasa 
vasorum. Future work will concentrate on further 
analysis of the origin of the absorption signal in depth 
and the Separation of these contributors. This will allow 
a greater understanding of the mechanism of oxygen 
transport across the vessel walls. Possible applications 
include monitoring and predicting the success of arterio- 
venous fistulae and optimisation of anastomosis 
procedure s. 
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INTRODUCTION 

The evolution of neurosurgery has been towards 
increasingly precise, delicate and safe surgical 
technique. Surgical robotics, which has the ability to 
eliminate tremor, reduce gain, and increase dexterity, 
therefore has the potential to greatly enhance surgical 
performance. A necessary prerequisite for the successful 
design and use of such robots is familiarity with the 
surgical instrument forces exerted by surgeons during 
neurosurgical procedures. The advent of surgical 
robotics has allowed, for the first time, the forces 
exerted during neurosurgical procedures to be routinely 
recorded [1]. The corollary is that expert performances 
can be analysed to determine the optimal force ranges 
utilised when performing robot-assisted neurosurgical 
procedures, allowing for the possibility of force limits to 
be set to improve surgical safety, and providing 
quantitative feedback to trainees to further their 
development. 

The aim of the present study was to measure the 
surgical instrument force exerted during robot-assisted 
neurosurgery by performing a human cadaveric study. 

MATERIALS AND METHODS 

An experimental rig was set up consisting of a platform 
for brain specimens, a Leica microscope (Leica 
Microsystems GmbH) to provide illumination and 
magnification, and a Quanser 6 Degrees-Of-Freedom 
(DOF) Telepresence System (Quanser Ine) for tissue 
manipulation and force measurements (See Fig 1). The 
robotic master-slave setup included a DENSO VP 
Series 6-Axis Articulated Robot and the control module 
(DENSO Robotics), a Gamma Multi- Axis ATI 
Force/Torque Sensor (ATI Industrial Automation) 
equipped with a 16-Bit Data Acquisition Board 
(National Instruments) to have accurate force 
measurements, and a High-Defmition Haptic Device 
(HD2) with the capability of providing 6 DOF 
force/torque feedback to the operator; this configuration 
allowed for a continuous force feedback of up to 20. ON. 
The robot utilized an open-architecture interface with 
the QUARC 2.2 Denso Robot block-set, alongside 
Simulink®, Matlab® and Windows®. There were no 
force or position scaling involved and the ratio of force 
feedback from the robot sensors to the haptic Controllers 
was set to 1:1. Calibration was independently verified 


using a Chatillon Digital Force Gauge with a capacity of 
ION and accuracy of 0.0 IN. 



Fig. 1 Experimental rig consisting of a platform for brain 
specimens, a Leica microscope, and a Quanser 6 DOF 
Telepresence System 

Two fresh brain specimens were utilized for the study. 
In each specimen, a neurosurgical trainee (HJM) was 
asked to coagulate the brain surface using bipolar 
electrocautery and then perform a sequence of simple 
procedures: (1) a Beaver Mini-Blade was used to incise 
the coagulated area, (2) the Mini-Blade was used to 
carry the incision to a length of approximately 30mm, 
and (3) a No 6 Penfield dissector was used to retract the 
brain approximately 5mm. These manoeuvres were 
carried out in the cerebrum (gyrus rectus, middle frontal 
gyrus, inferior temporal gyrus), cerebellum 
(hemispheres, vermis) and brainstem (midbrain, pons, 
medulla); and each was repeated twice on each side of 
the brain, unless it was too technically difficult to do so. 
In addition, the forces required to perform a corpus 
callosotomy and perforate the floor of the third ventricle 
were also measured. 

The Beaver Mini-Blade and No 6 Penfield dissectors 
were also used to perform sharp and blunt dissection of 
the circle of Willis respectively. An observer carefully 
took note of iatrogenic injury to neurovascular 
structures during dissection. In cases of uncertainty, we 
recorded injury as having occurred. 

Force vector data was generated each 0.2ms (5 KHz). A 
Matlab® program was developed and used to extract 
data each 1 0ms and record in an Excel spreadsheet, and 
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force vectors were summated. Statistical Software 
(SPSS 20.0.0) was then used to calculate the arithmetic 
mean, Standard deviation and maximal forces during 
individual procedures. The paired T-test was used to 
compare the surgical instrument forces exerted in each 
brain specimen, and on the left- and right-side of the 
brain. The ANOVA test was used to compare the forces 
exerted when performing different procedures, and 
between different brain regions; if significance was 
demonstrated post-hoc analysis was performed using the 
Student Newman-Keuls test. The unpaired T-test was 
used to compare the forces exerted when performing 
sharp and blunt dissection, and the manoeuvres that 
resulted in iatrogenic injury against those that did not. 
The threshold for significance was set at 5%. 

RESULTS 

In all, there was no significant difference in forces 
exerted between the two brain specimens, and between 
the left- and right sides of the brains. The mean force 
exerted when performing stab incisions (0.0 IN) was 
significantly less than the forces required to carry the 
incisions (0.09N) or retract brain (0.10N). The mean 
force exerted when manipulating the brainstem (0.09N) 
was significantly greater than that required in the 
cerebellum (0.05N) or cerebrum (0.05N). 

The mean, Standard deviation, and maximal forces 
exerted when performing sharp and blunt arachnoid 
dissection around the Circle of Willis are summarized in 
Table 1; manoeuvres are stratified according to whether 
they did or didn’t result in iatrogenic injury. 


Table 1 The mean, Standard deviation and maximum forces 
exerted when performing sharp and blunt dissection of the 
Circle of Willis 



No injury 
(n=34) 

Injury 

( n= 6) 

Sharp 

Arachnoid 

Dissection 

(n=28) 

Mean ± 

SD (N) 

0.07 ±0.08 

0.28 ±0.08 

Max 

(N) 

1.33 

2.50 

Blunt 

Arachnoid 

Dissection 

(n=12) 

Mean ± 

SD (N) 

0.15 ± 0.11 

0.68 ±0.32 

Max 

(N) 

2.04 

4.28 


The mean force exerted during blunt dissection (0.33N) 
and sharp dissection (0.09N) was significantly different. 
When performing blunt dissection the mean force 
during manoeuvres in which iatrogenic injury occurred 
(0.68N) and those in which it did not (0.1 5N) was 
significantly different. When performing sharp 
dissection the mean force during manoeuvres in which 
iatrogenic injury occurred (0.28N) and those in which it 
did not (0.07N) was also significantly different. 

DISCUSSION 

In this study we have successfully determined the 
surgical instrument forces exerted during a number of 


robot-assisted procedures on fresh cadaver brains. The 
measured forces varied significantly depending on the 
region of the brain and the manoeuvre performed. 
Moreover, blunt arachnoid dissection with a Penfield 
No 6 dissector was associated with greater force 
exertion, and a higher risk of iatrogenic injury, than 
sharp dissection with a Beaver Mini-Blade. 

Existing literature on the forces exerted during 
neurosurgical procedures is sparse. Howard et al 
measured the penetration forces on 2.5mm spheres and 
the drag forces on a 3mm ventricular catheter advanced 
20-3 0mm deep into the brain tissue of patients 
undergoing temporal lobectomy, and reported forces of 
0.08N and 0.03N respectively [2]. These findings are 
comparable to our own, supporting the notion that the 
forces necessary to manipulate brain tissue are low. 

Although the present study measured forces carefully 
using an extemally calibrated System, it has several 
limitations. First, the cadaver brains utilised 
undoubtedly had different tissue properties to living 
brain tissue, though fresh cadaver brains were used to 
ameliorate this. Second, the forces exerted during the 
simple procedures performed may not reflect those 
required during the complex technical manoeuvres of 
actual neurosurgical operations. Third, the surgical 
forces exerted during microsurgery are known to be 
greater in novices than experts [3], and the forces 
applied by the neurosurgical trainee in this study are 
therefore almost certainly higher than if a Consultant had 
performed it. 

In future studies we plan to address the aforementioned 
shortcomings by analysing the data generated by the in- 
vivo use of neuroArm, an MRI-compatible robot 
incorporating haptic feedback. In contrast to the present 
cadaver study, such data will be obtained in real 
patients, undergoing complex surgery, and performed 
by experts. 
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INTRODUCTION 

Solid-Organ-Disease is one of the most common 
diseases in China. Due to the excessive intraoperative 
blood loss, solid organ surgeries are usually complicated 
and time consuming. To assist this kind of surgery, a 
compact minimally invasive surgery (MIS) robot has 
been developed. Several magnetic sensors, manipulated 
by surgeon, are used to collect the Operation information, 
which is used to control the miniaturized slave arms. 
This feature makes the robot have little requirement on 
the Operation room. To further improve the Operation 
effectiveness, a coagulation device that can transport 
energy from the energy generator to the inner tissue of a 
solid organ through tiny incisions on patient’ s body wall 
was developed. The proposed technique was verified by 
an in- vitro experiment. 

Related work. For solid organ resection, a specially 
designed tool, developed by AngioDynamics, has been 
used in open surgeries and good results are obtained [1]. 
With the rapid development of MIS, a laparoscopic 
bipolar resection device has also been developed by the 
Company to reduce the operative trauma [2]. However, 
the dexterity is limited. The advanced robotic 
technology in MIS has already leaded a momentous 
change in surgery [3]. In this Situation, we expected to 
make the solid organ surgeries easier by integrating 
advanced robot technology and existing technology as a 
whole. And a prototype has been developed in the lab. 

MATERIALS AND METHODS 

If the tissues on a solid organ are coagulated first, then 
little bleeding would occur when operates on the 
coagulated area. Energies, such as electric energy, 
microwave, radio frequency, and ultrasound, have been 
widely used in operations to coagulate tissues to reduce 
blood loss. This effective method is adopted to develop 
the robot based dexterous coagulation device. 

1) Brief Introduction about the MIS Robot 
The newly developed MIS robot System is shown in Fig. 
1. The slave robot, with small size and light weight, 
consists of three arms, each of which can pivot around 
the incision set on patient’ s body in three directions. 
The collapsible structure makes the robot easy to 
transport. Several 3D Guidance trakSTAR sensors 
(Ascension Technology Corporation) embedded in 


different handles are used to collect the Operation 
information of the surgeon’ s hands. The no-mechanical- 
structure design of the master System further reduces the 
space that occupied by the whole robot. 



Fig* 1 The newly developed MIS robot. 


2) Instrument Design 

The instrument for solid organ surgery under minimal- 
invasive condition is shown in Fig. 2. The diameter of 
the shaft is 8mm. The pretensioned Steel wires are used 
to transmit motion/force from the driving units to the 
end-effector. The needle-like end-effector is designed to 
minimize the blood loss during the piercing process and 
to decrease the adherence of the solidified tissues. Four 
needles, two connected to positive terminal and the 
others connected to negative terminal of the energy 
generator, are used to enhance the coagulation effect. 



Fig. 2 Coagulation tool for robot-assisted solid organ 
surgery. 


The 2-DoF instrument combined with the 3-DoF slave 
arm can make the needles be pierced into tissues in any 
directions. The instruments can be quickly attached or 
detached from a robotic arm through the surgical 
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adaptor. This function is essential as the instrument has 
to be cleaned when too many scabs are adhesive to the 
needles. 

3) Operation Method 

The Operation method of the proposed bloodless cut 
device is shown in Fig. 3. Surgeon need to distinguish 
the healthy tissues and the diseased ones and determine 
the cutting line based on diagnose results. Then the 
surgeon manipulates the handles to control the slave 
arms to position and orientate the needles. After that, 
the surgeon will control the needles to insert into the 
organ along the needles’ direction carefully. When 
needles reach the desired depth, a signal is given to the 
energy generator to release energies. At the same time, 
needles are controlled to do automatic reciprocating 
motions, which can enhance the coagulation effect, as 
well as decrease the adherent of solidified tissues. 
Repeat the above process until all the tissues around the 
cutting line are solidified. Then, assistant can install a 
scalpel for the main surgeon to separate the diseased 
tissues from the organ. Sometimes, due to the limited 
length of the needles, one or more coagulation- 
separation processes are needed to complete the 
Separation. 


Fig. 3 The Operation method of the proposed method. 

4) Experiment 

An in-vitro experiment has been carried out. The 
experiment setting is shown in Fig.4. A fresh pig’s liver 
harvested within 30 minutes was used in the experiment. 
Two slave robot arms, one for manipulating a forceps 
and the other for manipulating the coagulation device, 
are controlled by the operator to do ‘operations’ on the 
liver. Scene of the Operation site was provided by the 
endoscope held by the third arm. An electrosurgical 
generator was applied in this experiment. 

RESULTS 

It took less than 8 minutes to coagulate the tissues along 
the line marked on the liver. The length of the 
coagulation area is 13cm. Figure 5(a) shows the whole 
coagulation procedure. The coagulation effect was 
checked after dividing the liver into two parts along the 
cutting line, as shown in Fig. 5(b). If s clear that the 
section was separate into coagulated area and normal 
area. For the coagulated area, little bleeding may occur 
when doing operations in a real solid-organ surgery. For 


the normal area, another coagulation process is needed 
after the coagulated area opened. 



Fig.4 The in-vitro experiment setting. 



Fig. 5 The coagulation procedure and the experiment results. 

DISCUSSION 

To improve the effectiveness of the solid organ 
surgeries, this work transplants the routine technology 
in open surgery to MIS without losing the Operation 
dexterity by means of robotic technology. A MIS robot 
based instrument that can transport energy to organ to 
coagulate tissues is proposed. The effectiveness of the 
proposed device was verified by an in vitro experiment. 
It can be concluded that the technique has great 
potential in clinical use. Animal experiments, however, 
are needed to carry out for further development. 
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INTRODUCTION 

Surgical robotic Systems are safety critical System, i.e. 
Systems where errors might lead to the endangerment of 
human life, to substantial economic loss, or to extensive 
environment danger [1]. Nevertheless, currently, no 
safety Standards for robotic surgical Systems exist, but 
other Standards for robotics are established, for instance 
the ISO 10218:2011 represents safety requirements for 
industrial robots, ISO 13485:2003 Standard specifies 
requirements for a quality management System for 
medical devices and EN ISO 14971:2012 represents the 
application of risk management System for medical 
devices. Indeed, defming safety for surgical robots is 
not trivial, since the concept of safety in surgery is 
strictly Operation and patient dependent [2] and involves 
both the aspects of safety for human-interactive robots 
[3] and surgical aspects [4]. Moreover, safety depends 
in a first instance on reliability, i.e. a System has to 
counteract to failures keeping its behavior inside a 
region of performance that guarantees the reliability of 
the actions performed. Failures are caused by faults, that 
are deviations of the System structure or parameters 
from the nominal Situation [5], [6]. We aim at achieving 
a fault analysis which consists in the detection of 
potentially dangerous situations, potential failures and 
the defmition of hazard risks during the whole life cycle 
of a surgical robotic System. This serves as a basis for 
fault detection and safety verification procedures. 

FAULT AND SAFETY ANALYSIS 

We introduce a procedure for fault analysis in surgical 
robotic Systems and determine some strategies for 
further verification of safety in such Systems. When 
dealing with faults in a surgical context, it is necessary 
to distinguish between human errors [7] and technical 
faults. Usually human errors lead to severe damage to 
people and tools, with liability and legal implications. 
For these reasons, we are not taking them into account 
in a first instance and focus on the technical faults. Our 
objective is to obtain a System that will be able to 
reconfigure itself after a fault occurrence respecting 
some safety constraints: no risks for the human beings 

This research has been supported by the coordination action 
EuroSurge (Grant no. 288233) funded by the European Commission 
in the 7th EC framework pro gram. 


in the operating room (including medical staff and 
patient) and correct achievement of the surgical task. 

To this end, we have to point out that a distinction 
exists between the fault analysis and the safety 
evaluation. Performing a fault analysis is a first Step to 
verify if the System can be designed in a fault tolerant 
way, i.e. if the System will be able to achieve the desired 
task even if a fault occurs. Whereas the safety 
evaluation is aimed at avoiding that the System (whether 
reconfigured after a fault or not) still remains in a safe 
region of performances, given in the specification 
design phase (see [8] and references therein). Our 
procedure first assures fault tolerance, by analysis of the 
possible faults occurring in a surgical robotic System 
and their effects propagation. Then we will perform an 
evaluation of the safety requirements, through 
specifications obtained from the ontological description 
of the surgical task. The state of the art in fault analysis 
includes two main methods: Fault Tree Analysis (FTA) 
and Failure Mode and Effects Analysis (FMEA). FTA is 
used on the overall System to check the fault 
propagation in the different functionalities of the 
System. The fault tree represents the connections 
between the different failure modes, where the leaves of 
the tree are the faults on components. Generally, the 
complementary Version of the fault tree is a 
functionality tree and each basic functionality can be 
associated to a component of the System [9]. FMEA is, 
instead, an approach that Starts from the components 
and is not able to show the fault propagation, but it is 
used to assess the risk of each possible failure [10]. We 
will use the FTA to determine what functionalities of 
the overall System will be affected by component faults. 
This first step is needed to verify that the System is able 
to achieve its goal even after the occurrence of a fault. If 
not, then it is necessary to modify the overall System 
and perform a new FTA. Next step consists in assessing 
the safety of the surgical robotic System. To this end, 
safety specifications are designed, based on the 
ontological description of the surgical task and on 
technical and medical knowledge about the Operation. 
Then a FMEA is performed, taking into account that a 
surgical Operation can be divided into three phases: pre- 
operative, intra-operative, post-operative. Each phase 
has its own specifications in terms of safety and FMEA 
will be performed in each phase, with associated risks of 
failure. To respect safety requirements, the final step 
consists in making the System able to counteract to 
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failures and perform a formal verification of the Overall 
System. 

Here we present the first part of the procedure, 
associated to FTA, for a simple case study introduced in 

tu]. 



Fig. 1 Functionality tree of the System with one tracker 

Our System is composed by a manipulator with a 
puncturing tool moving from point A (stand-by 
Position) to point B (located on the patient skin) and 
controlled in position. The end effector position is 
monitored by a tracker with 10 cameras that follows a 
marker on the end effector. The estimated position is 
given by triangulation and sent to the Controller. The 
functional tree of the System is reported in Fig. 1. The 
target point is point B. The tracker functionality consists 
in having, at each sample time, an estimate of the real 
position of the end effector. 



Fig. 2 Fault tree corresponding to the functionality tree in fig. 
1. (FC=Faulty Camera, PL= Power loss, SE= Software error) 

The corresponding fault tree is present in Fig. 2. We 
focus on faults on the tracker. From the fault tree in fig. 
2 it is straightforward that, if the tracker is not working 
(we lose the tracker functionality), the System is not able 
to reach the target anymore, so the System is not fault 
tolerant. To achieve fault tolerance it is necessary to add 
hardware redundancy, for example using a backup 
tracker, as shown by the functionality tree in Fig. 3. 
Since the backup tracker might have different properties 
from the main one, a safety specification might be that 
the System adapt to the backup tracker when this is 
activated, for example slowing down if the backup 
tracker has a bigger sample time, in order to reach 
safely the patient skin. Of course other safety 


specification might be verified. The safety evaluation 
from ontology and FMEA are work in progress. 



Fig. 3 Functionality tree of the System with two trackers 
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INTRODUCTION 

Robot-assisted minimally invasive surgery (RMIS) has 
been widely applied in recent years [1]. It has many 
benefits over traditional minimally invasive surgery 
(MIS), such as an improvement of dexterity and 
enhanced 3D vision. However, the sense of touch is lost 
- this is considered as a significant drawback of RMIS. 
In order to identify the tumour location and its 
boundaries in cancer excision, real-time tissue probing 
methods using force-based sensing or tactile-based 
sensing have been proposed. The perception of direct 
force or tactile feedback (pressure information 
distribution) is intuitive. However, the application is 
limited by the relatively high cost and control 
complexity of tactile actuators. Altematively, low-cost 
visual cues can be used to compensate for the lack of 
haptic Sensation. Graphical overlays showing the 
material property distribution have been presented [2], 
[3]. However, the graphical overlay of real-time 
stiffness data on camera images may affect the clarity 
and quality of the original camera image. Further, it is 
time consuming to investigate a large tissue area using 
uniaxial indenting based on separate-point probing. 
Continuous scanning and indentation depth 
measurement would improve the efficiency and the rate 
of stiffness estimation for larger areas [4]. In this paper, 
we propose a visual stiffness feedback method for soft 
tissue palpation in RMIS, operating in real time. The 
novel feature of our method is a dynamically updated 
soft tissue stiffness map that is generated from 
continuous rolling indentation and is overlaid on a 
separate virtual soft tissue surface. We demonstrate the 
feasibility of the interface in a master-slave tele- 
operation configuration. 

MATERIALS AND METHODS 

The real-time visual stiffness feedback method in a 
master-slave teleoperation configuration consists of the 
following main components: telemanipulators (a master 
robot and a slave robot), a vision System and a visual 
stiffness display. Fig. 1 shows the frame of the System 
design. Real-time camera images of the palpation site 
are provided. Sensor measurements including force and 
Position are transmitted from the slave robot side to the 
master side to enable the virtual soft tissue surface 
reconstruction and visual stiffness feedback. 
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Fig. 1 The frame diagram of real-time visual stiffness 
feedback for soft tissue palpation in a telemanipulation 
environment 


The virtual soft tissue surface is displayed on a graphical 
interface using a mesh of triangles, whose vertex 
positions are acquired from a scan of the tissue contour. 
The indenter is moved manually across the tissue in such 
a way that it is just in contact with the tissue surface 
during the contour scan with the help of force feedback. 
Tissue deformation of the virtual soft tissue during 
palpation is shown in real time by using a geometrical 
deformable soft tissue model considering the influence 
of the indenter diameter based on predefmed finite 
element modelling; the details of this model are 
presented in [5]. A real-time soft tissue stiffness 
estimation method is used. If a small indentation is 
applied and using a rigid spherical indenter, the elastic 
modulus of tissue can is 

E = m i+O, 

8 d Jrd. 

in v in 

where E is the elastic modulus, / is the tissue reaction 
force normal to tissue surface, r is the radius of the 
spherical indenter, d in is the indentation depth and v is 
the Poisson ratio. Since we assume soft tissue 
incompressible, v is set to 0.5 here. The indentation 
depth is calculated using the distance between the 
indenter position and the nearest triangle planar of the 
original contour. Using this indentation depth calculation 
method, not only the indentation depth on a planar 
surface can be acquired, but the indentation depth can 
also be calculated on uneven surfaces. In this real-time 
soft tissue stiffness estimation method, the acquired 
indentation depth and reacting force are the two inputs. 
The calculated elastic modulus is the output. Next, a 
real-time visualization of stiffness distribution is created 
using estimated stiffness data. The estimated elastic 
modulus, E , is stored jointly with its palpated location. 
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Then, this value is converted to an RGB value by 
interpolating between the colour representatives of 
current minimum and maximum stiffness values. Blue 
colour (R=0, G=0, B=255) represents the minimum 
stiffness and red colour (R=255, G=0, B=0) represents 
the maximum stiffness. These pairs of RGB values and 
palpated locations are used to update the displayed 
stiffness map dynamically. 


To demonstrate the feasibility of the feedback method, 
an experiment was conducted. As shown in Fig. 2, a 
PHANToM Omni (SensAble Technologies Inc., MA, 
USA) and a FANUC robot arm (FANUC Corporation, 
Japan) were used as the master and slave robot, 
respectively. A Microsoft Kinect camera was used to 
obtain a live camera image. The master robot, the 
camera image, and the visual stiffness display were 
located at the user’s side for remote manipulation. On 
the master side, an operator held the Stylus of the 
PHANToM Omni to command the slave robot to palpate 
the silicone phantom organ. By looking at the live image 
and visual stiffness display on a soft tissue model 
generated from a tissue contour scan, the operator was 
able to acquire the stiffness distribution. The palpation 
trial was conducted continuously using the rolling 
indentation probe [4]. 


Camer 



Slave 

Robot 

Arm 

Stiffness 

Probe 


Silicone Phantom Organ 


Fig. 2 Experimental configuration of slave side hardware, 
including a slave robot arm, a silicone phantom organ and a 
camera 



Fig. 3 The silicone phantom organ with three embedded 
sphere nodules (A, B, C); the diameters of nodules are 10 mm, 
8 mm, and 6 mm. the depths from sphere top to silicone 
surface are all 6 mm. 


RESULTS 

Fig. 4 shows one picture taken during a palpation trial. 
A virtual soft tissue with a dynamically updated 
stiffness map was provided on a graphical interface at 
the master side. Three hard nodules were found as 
indicated by the coloured stiffness map, which matched 
the locations of the three embedded nodules in the 
silicone phantom. The stiffness levels of those three 
positions also matched the nodule sizes (A>B>C). The 


stiffness map helps the operator to have an informed 
view of the stiffness distribution. 



C (6.8 kPa 


Hardest 

A 

E 

▼ 
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(a) (b) 

Fig. 4 One picture (a) taken during palpation over the surface 
of the silicone phantom tissue. Subfigure (b) is the stiffness- 
colour mapping. In (a), the locations of the three embedded 
nodules (A, B, C) are displayed on the stiffness map. 


DISCUSSION 

According to the experimental evaluation using a 
silicone phantom with embedded artificial tumours, the 
proposed real-time visual stiffness feedback method for 
soft tissue palpation provides a possible solution for 
intra-operative tumour localization in RMIS. Future 
work will be focussing on palpation trials on soft tissue 
with uneven surfaces. The Kinect camera will be used to 
obtain point cloud data and reconstruct the tissue surface. 
The method has the potential to be translated to 
prostatectomy, breast surgery and gynaecology to help 
surgeons to archive a more accurate detection of 
tumours. 
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INTRODUCTION 

The detection of tissue surfaces during endoscopic 
surgery has the potential to allow patient-specific 
pre-operative images to be used for surgical guidance. 
The most commonly attempted methods include 
stereoscopy, SLAM, shape from shading/motion, 
time-of-flight, and structured lighting (SL) [1-5]. Of 
these, SL and time-of-flight have the advantage of not 
being dependent on tissue texture. Endoscopic 
implementations of SL are difficult due to requirements 
for high light levels and unambiguous feature detection, 
although there have been some recent reports in the 
literature [6-8]. 

In previous work, we have demonstrated an SL 
System that uses a supercontinuum fibre laser together 
with an optical fibre probe (diameter 1 .9 mm) to create 
an array of Spots that are each at a unique wavelength (~5 
nm bandwidth) [6]. This approach has many advantages, 
including the high brightness of the pattern, detection by 
a Standard endoscope camera, low sensitivity to tissue 
pigmentation, small diameter and flexibility. A beam 
chopper switches rapidly between SL and white light 
modes, providing video rate display for the surgeons [7]. 
A central part of the process to reconstruct the tissue 
surface using this System is the identification of the 
coloured Spots. Previously an RGB -to- wavelength 
conversion was used to estimate the spot centres and 
identify them. In this paper we present a faster, more 
accurate and robust method to detect centroids as well as 
boundaries of different projected multispectral laser 
spots and to achieve the unique identification of each. 
The other stages in the imaging procedure (calibration, 
triangulation and reconstruction) as well as a description 
of the hardware used are described in previous 
publications [6, 7]. 

MATERIALS AND METHODS 

A schematic of the algorithm is displayed in Fig. 1. The 
RGB images collected by the camera are firstly 
converted into images in HSV space to separate intensity 
and hue information for the following steps. 
Thresholding is used to suppress background noise and 
generate a convex hüll that encloses the spot pattern, and 
then a binary image created by a lower threshold is 
combined with the convex hüll, generating a mask 
representing the region of interest (ROI). The intensity 
and hue maps are filtered by Wiener and Gaussian filters, 
to suppress noise and enhance the intensity peaks of the 


spot centres respectively, followed by histogram 
equalization. Regional maxima of the intensity map are 
then calculated, using the extended H-maxima transform, 
which correspond to the bright central areas of each spot. 
These are treated as initial seeds for the region growing 
procedure. The criteria for this are established using 
ränge s defmed by the mean and variance of all pixel 
intensity and hue values in each seed. The grown result is 
evaluated by average spot size and similarity of hue 
value in order to merge small spots that are in contact. 
Spots identified as too small or large are improved by 
further region growing using ‘tighter’ or Tooser’ growth 
criteria. Finally, morphological transforms such as image 
opening and closing are used to refine the shapes and 
boundaries of grown spots. 
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Fig. 1 (a) General 3D reconstruction procedure. (b) Example 
of spot detection using region growing. 

After detection each spot is identified using an 
algorithm that incorporates both colour and spatial 
information in the pattern. A Delaunay triangulation is 
performed on a reference image where all spots have 
been detected. The normalised RGB values of the 
vertices of each triangle are then recorded. For a 
particular test image a spot of interest will be a node for 
several surrounding triangles. To identify each spot in 
other images the spot with the highest number of 
matching neighbours is chosen. This procedure is done 
by matching triangles based on the similarity of their 
interior angles (within a defmed threshold) and the 
Euclidian distance between their nodes’ RGB values. 
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RESULTS 


Seven SL images from a calibration object and ex vivo 
porcine tissue (three from a white plane, two from liver, 
one from kidney and one from heart) were tested using 
the spot detection method. The result is compared with 
manually-identified Spots, evaluated by visual inspection 
and quantified in terms of sensitivity and precision. The 
results are shown in Fig. 2 and Table 1. 




Fig. 2 Images of detected centroids and spot boundaries 
overlaid on RGB images of calibration and porcine tissue 
surfaces. (a) White plane, (b) Liver. (c) Heart. (d) Kidney. 


Table 1 Spot detection results for seven images. Sensitivity 
indicates the fraction of spots correctly detected; Precision is 
the fraction of correct detections in all detected spots. 


Sensitivity 

Precision 

Surface 

0.9417 

0.9912 

White Plane image 1 

0.9667 

1.0000 

White Plane image 2 

0.9508 

0.9831 

White Plane image 3 

0.7672 

0.9674 

Liver image 1 

0.7500 

0.9355 

Liver image 2 

0.7297 

0.9529 

Heart 

0.6102 

0.8675 

Kidney 


The results shown in the above table indicate high 
sensitivity and precision for images of the white plane. 
However, sensitivity is lower for soft tissue. This is 
mainly caused by strong absorption at the blue end of the 
Spectrum due to haemoglobin. Given that global 
Parameters are adopted in the local maxima searching 
procedure, some blue spots are missing. For darker spots 
whose intensities are similar to those of the background, 
boundaries detected by region growing are also 
inaccurate. Furthermore, large distortions on some parts 
of tissue surface also result in severe spot shape 
deformation, introducing more detection error. 

Figure 3 shows the results of the spot Identification 
algorithm between test and reference images in a 
calibration dataset. The performance was evaluated by 
analysing the number of correctly identified spots (true 
positives, TP) and incorrectly detected spots (false 
positives, FP). The probability that a spot was labelled 
was calculated using P L =[TP+FP]/TP along with the 
probability that each labelled spot was labelled correctly 
(P C /l=TP/[TP+FP]). High P L (89.8%) and P c/L (98.6%) 
values were obtained. Robustness was tested by 
removing spots at random and repeating the matching. 
As spots are removed P L decreases but P C /l remains 
stable close to 100%. 



Fig. 3 Spot Identification, (a) Reference showing numbered 
spots and triangulation. (b) Test image with matched triangles. 


DISCUSSION 

Initial results from a multispectral spot detection 
algorithm have been presented, indicating ability to 
detect spot boundaries on tissue. Identification using 
colour and spatial information has been demonstrated 
that is robust to occlusions and low brightness. The 
results show that the System could potentially be used in 
lesion detection and to aid rigid and flexible endoscopy if 
adapted to ‘ front- viewing’ Systems. 

Future work will involve evaluation of the robustness 
of these algorithms in the presence of varying tissue type 
and morphology. Besides, this boundary detection 
method introduces a possibility for denser 3D 
reconstruction and adaption with near infrared cameras. 
Hardware modifications are also planned to alter the 
distribution of wavelengths of the spots and record high 
dynamic ränge images using different camera exposure 
times. Real-time computation would also allow a rapid 
reconstruction of the surface. 
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INTRODUCTION 

Minimally invasive surgery (MIS) has evolved from 
traditional laparoscopy, which involves the surgeon 
using hand held tools through small incisions on the 
patient’ s body, to robotically assisted (R-A) surgery, 
during which the surgeon remotely operates articulated 
instruments attached to the end of robotic arms. 
Advances in the design, articulation and flexibility of 
the instruments have added to the popularity of R-A 
MIS [1]. Nevertheless, the way that the instruments are 
controlled affects not only their efficacy, but also the 
ergonomics and the learning process for the surgeon. 

In [2], we have presented a concept for hand-like 
instruments, each carry ing an articulated 3-fmger 
System which imitates the surgeon ’s fingers’ exact 
movements. The anthropomorphic System design aims 
to reduce the ‘cognitive gap’ between the way that 
instruments are manipulated and the surgeon ’s natural 
hand movements. The master Controls of the Da Vinci 
Surgical System [3] manipulate a simplified gripper 
attached to a 3-DOF wrist. In a very similar way, the 
Phantom Omni haptic device is used to control surgical 
instruments in various other designs of MIS instruments 
[4]. If the instruments to be controlled are more 
complex, more complex master devices are required. 

Systems for motion capturing of hands span from on- 
the-hand hardware with finger position capturing, such 
as data gloves and exoskeletons, to extemal imaging 
Systems based on intensive image processing and often 
covering a limited field of view. Data gloves, used for 
similar tasks, are expensive and generally lack 
durability [5]. Besides being non-adjustable and 
requiring calibration for each user, data gloves do not 
offer detailed joint tracking. In [6], a wrist- worn real 
time hand tracker is proposed, which avoids burdening 
of the hand with extra load. Limitations include 
occlusions resulting from overlapping fingers. Besides 
finger tracking, wrist rotation and bend tracking is very 
important especially during surgical tasks. However, 
these cannot be modelled using a wrist-wom tracker or 
a data glove. 

Consequently, we report here on the development of a 
lightweight and adjustable hand exoskeleton that can 
sense movements of the surgeon’ s finger’ s joints and 
translate it to movements in the joints of the instrument, 
such as the one we presented in [2]. Most exoskeletons 
in the literature are aimed at actuation of hands or arms 
[7], often ending up being bulky and heavy. By 


removing all the motors and encoders, the design can be 
simplified. The following sections present the design of 
an initial prototype of a hand exoskeleton for control of 
the MIS instrumentation in the MIS concept described 
in [2]. 

MATERIALS AND METHODS 

The proximal interphalangeal (PIP) and distal 
interphalangeal (DIP) joints of the fingers are hinge 
joints capable of only flexion and extension (1 DOF 
each). The metacarpophalangeal (MCP) joints at the 
base of the index and middle finger, however, are saddle 
joints, and hence capable of abduction and adduction (2 
DOF). The thumb can also be modelled by having two 
1-DOF (interphalangeal-IP and metacarpo-phalangeal- 
MP) and one 2-DOF (Carpometacarpal-CMC) joints. 
The exoskeleton was designed according to this layout, 
so that it can follow the fingers’ natural motion. 

The angle of each joint is measured using two types of 
hall-effect sensors (MLX90316 and MLX90333, 
Meiexis, Belgium) measuring 1 DOF and 3 DOF. The 
1-DOF sensor can give absolute angular position of a 
small magnet located parallel to the sensor in a rotary 
type joint. The 3-DOF sensor senses the magnet 
position anywhere in its surroundings, being suitable for 
a ball type joint. In order to simplify the exoskeleton 
design, the PIP and DIP of the index and middle fingers 
were considered coupled (as they are in the human 
hand). Therefore, instead of having different sensors for 
each joint, the exoskeleton carries only one sensor for 
the PIP joint, while the position of the DIP is calculated 
by the relationship between the PIP and the DIP given 
in [8]. The whole structure comprises seven sensors in 
total: three 2-DOF (MCPs and CPC) and four 1-DOF 
(PIPs, IP and MP). 



Fig.l CAD drawing of the exoskeleton assembly 
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RESULTS 

The computer-aided drawing with the main exoskeleton 
components is shown in Figure 1. Each exoskeleton 
joint is fastened to the hand with a flexible attachment. 
The joints are connected to each other via adjustable 
links. The MCP and CPC joints were designed as ball 
joints in order to reduce the bulkiness and the 
complexity of the component. The sensors are attached 
at the side of each joint on the non-contacting parts, as 
shown in Figures 2 and 3. 


Fig. 2 Ball joint prototype and ränge of motion calculation 

The typical ranges of motion of each joint in degrees 
are: 0-90° for the MCP, 0-110° for the PIP and 0-70° for 
the DIP. To ensure unrestricted motion, the ränge of the 
ball joint was set to be at least 90°. From the parameters 
in Figure 2, the ränge of the ball joint can be calculated 
as: 

26 = 2 (90 - ^ ^ — $ J = 2 (90 - sin- 1 — - sin” 1 
where R 2 = 6 mm, d = 3 mm and r = 1.5 mm. 

^ D sensor 


Fig. 3 Revolute joint and double threaded adjustable link 

The joints are connected to each other via double 
threaded links (Figure 3). One side has a left-hand 
thread, while the other has a right-hand thread so that, 
by turning the link, it can be extended (clockwise) or 
shortened (anti-clockwise). 

DISCUSSION 

The design for a sensory hand exoskeleton presented in 
the previous section meets the design requirements. The 
double thread links not only make the mechanism 
adjustable to a variety of hand sizes, but also allow the 
exoskeleton to be lightweight (approx. 130 gr), without 
added material for modifications as in [7]. Furthermore, 
each of its joints covers a ränge of motion similar to, or 
greater than, that of a human hand, which ensure s 
natural unrestricted hand motion and comfort. The 
whole structure is as compact as possible and it has 
twelve DOF, ten of which are actively sensed while the 
other two can be calculated from the neighbouring 
joints. The next Version of the exoskeleton will include 
sensors for wrist motion tracking, in contrast to the 
methods used in [5-6], where this would not be possible. 

The exoskeleton was fabricated using 3D printing 
(NanoCure, Envisiontec, Germany). Figure 4 shows part 






of the exoskeleton fitted on the side of the index Finger 
and the electronics attached to the wrist. Sensor 
Processing electronics will be placed in a compact box, 
the size of a hand watch. 



Fig. 4 Testing the exoskeleton on one Finger 


In the first phase of our investigation, the surgeon will 
explore use of the exoskeleton to manipulate virtual 
objects. The sensory electronics will be connected to a 
PC via USB and the output of the sensors fed into a 12- 
DOF hand kinematic model. The user will wear the 
exoskeleton and the movements of his/her fingers will 
be simulated in real-time. This will form the basis of a 
surgical Simulation environment, where the surgeon will 
be able to test the concept of Controlling hand-like 
instruments as described in [2]. The accuracy of the 
structure needs to be tested and the teleoperation 
suitability will be evaluated in our future work. At that 
stage, the exoskeleton will be connected directly to two 
corresponding surgical hands. 
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INTRODU CTION 

This paper introduces the Software architecture of our 
robotic device for performing Ophthalmie surgeries with 
reduced tremors and increased precision. This robot not 
only improves current clinical procedures but also opens 
new avenues for treating Ophthalmie ailments. For 
example, Retinal Vein Occlusion currently has no 
generally approved surgical treatment. 

The robot has been designed to be as small as an 
average human hand and it weighs 315 grams, which 
can easily be mounted on the patient’ s head. It 
comprises five piezo drives with 0.5pm precision 
encoders. These drives are arranged in a hybrid parallel- 
serial configuration. Customized power electronic 
Controllers facilitate low level control of the drives 
through saw tooth type signals. The surgeon closes the 
loop through a visual feedback from an Ophthalmie 
microscope. He or she Controls the velocity of the tool- 
tip attached to the robot in Cartesian space by using a 
six degree of freedom (DOF) space mouse [4] (see 
Fig.l). It is worth mentioning that the tooltip velocities 
in Cartesian space are mapped to joint space velocities 
(considering the robot ’s Jacobian matrix). The velocity 
of each piezo drive is controlled by changing the 
frequency of the saw-tooth signal. The challenge is to 
keep the time delay between the motion of the surgeon 
and the motion of the robot as small as possible. 
Additionally, for seamless Integration into ORs, the 
setup must be compact. 



Fig. 1 Ex-vivo experiment on a pig’s eye. The current 5DOF 
robot is controlled using a space mouse. (1. Microscope, 2. 
Robot, 3. Space Mouse, 4. Power Electronic boards, 5. Robot 
Controller (Embedded), 6. Input Device Controller 
(Embedded), 7. Ethernet Switch) 


In this paper we propose a middleware driven 
architecture using an embedded platform. The 
architecture Supports hard real-time Operation along 
with plug-and-play Operation through low level device 
drivers. [4] and [5] present robotic Systems for steady 
and micro manipulation. [8] present a component driven 
architecture for integration of robotics integration. In 
this paper we mainly focus on middleware architecture 
for embedded Systems. 


SYSTEM ARCHITECTURE 



Fig. 2 System Architecture 


We have employed embedded middleware based 
architecture to facilitate a distributed Operation and easy 
integration. It allows the representation of process data 
at a higher level of abstraction and thereby facilitating 
easy integration and seamless data sharing within a 
unified ecosystem. It also provides an API for user 
space application development. 

The main distributed components (see Fig. 2) are the 
input device Controller and the high level robot 
Controller. Each component is hosted on an independent 
Raspberry pi board [3]. The board consists of an ARM 
700 MHz processor running a Debian Linux Kernel 
patched for hard real-time performance. The Xenomai 
[i] real-time patch is used which allows the 
development of hard real-time applications from user 
space. It supports USB for I/O and Ethernet for 
networking purposes. 

The middleware in use is the embedded extension of 
Internet Communication Engine (ICE) [2] and is called 
IceE. It uses TCP/IP as the communication medium and 
is platform and programming language independent. 
IceE allows the user with a message scripting language 
called Slice [2], through which the user can represent 
the data in the form of message objects and define 
interfaces for transporting these messages. Thereafter, 
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the slice tools automatically generate source code for 
the message interfaces in languages such as C++, Java, 
C#, Obj C, Python, Ruby and PHP. We employ the C++ 
development framework. In our architecture, a message 
interface is written for the data acquired from the space 
mouse by the input device Controller. It consists of the 
translation and rotation data specified in a structure 
along with the interface function for transporting them 
over the network. Fig. 3 illustrates the message object 
and interface. A publish/subscribe mechanism is used 
wherein the input device Controller hosts an Ice- 
Publisher. The high level robot Controller implements an 
Ice-Subscriber to read the published message containing 
the data from the space mouse. This data is further 
processed and mapped to the robot joints through the 
robot inverse kinematics. 

#ifndef INPUT^ICE 
#def ine INPUT^ICE 

module input 
{ 

struct InputDevice 
{ 

double translation x; 
double translation_y; 
double translation z; 
double rotation x; 
double rotation_y; 
double rotation z; 

} ; 

interface Inputlnterf ace 
{ 

void sendlnputData ( InputDevice input data); 

} ; 

} ; 

#endif 

Fig. 3 Ice Message Object and Interface 

Both the input device and the robot Controller 
components contain a hard real-time process using the 
Xenomai Library and the real-time patched Linux 
Kernel. In the input device application the data from the 
space mouse is acquired using the spacenav library [4]. 
This data is then packaged in the form of the ICE 
message object and published using the message 
interface. The high level robot Controller component 
reads this data using the subscriber and processes it for 
tremor reduction and motion scaling. Thereafter, the 
data is mapped to the robot joints through inverse 
kinematics. The robot itself is controlled in the velocity 
mode. A low-level device driver has been developed in 
Order to control the individual piezo drives of the robot. 
The driver sends ASCII based motion and feedback 
commands to the drives through a Serial over USB 
protocol. This data is received by the Firmware hosted 
on the low-level-robot control board which implements 
the power electronics. The complete control loop runs in 
a hard real-time process hosted by the high-level robot 
Controller. Fig. 4 illustrates the robot Controller 
consisting of the high level and low level components. 

RESULTS 

The first ex-vivo clinical experiments using the Setup, 
which is seen in Fig. 2, were performed successfully 
utilizing the embedded real-time architecture. The input 
device motions were mapped to the slave robot through 
velocity based control for intuitive Operation. Due to the 
compact size and simplicity of the hardware, integration 
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Fig. 4 Robot Controller 




of the Setup in the Operation room was trivial. The 
device was operated by Ophthalmie surgeons. The real- 
time capabilities of the computing environment ensured 
a consistent cycle time of 1 msec in the input Controller 
and the high level robot Controller threads. Furthermore, 
the real-time mapping of the surgeon’s motion to robot 
motion provided intuitive Operation at the scale of 
microns. It also significantly increases the robotic 
Operation safety while the reaction of the robot to 
human commands is fast. In the next Step a 3D OCT 
(optical coherence tomography) device is going to be 
integrated to the setup based on the same architecture. 
OCT information will be then used as additional 
intraoeular feedbacks for the surgeons. 

This works is supported by the Bundesministerium für 
Bildung und Forschung (BMBF; 16KT1223). 
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INTRODUCTION 

The clinical uptake of robot-assisted laparoscopic 
surgery has predominantly been for urological 
indications in adult patients, with several of these 
procedures becoming established as Standard of care. In 
paediatric surgical specialties, adoption of this surgical 
technology was initially slow and cautious but recently 
has seen growing enthusiasm. Adult and paediatric 
urological practice unavoidably share much similarity 
due to common anatomic regions of focus, however 
there are also many unique differences. For example, 
the proportion of congenital and acquired pathology 
encountered by a paediatric urologist is heavily skewed 
towards the former, and vice-versa for an adult 
urologist. The role of robot-assisted surgery for 
urological indications in children remains unclear. This 
paper examines the seven-year experience of a single 
surgeon following establishment of a robotic surgical 
Service for paediatric urology. 

METHODS 

All children undergoing robot-assisted urological 
surgery between March 2006 and March 2013 were 
included. One surgeon (A.S.N) performed all cases in a 
single Institution, using the da Vinci® Surgical System 
(Figures lb, lc). All clinical data was prospectively 
recorded in an electronic database. 

RESULTS 

There were 175 patients involving 185 procedures. The 
ränge of procedures performed included pyeloplasty 
(88), nephrectomy (27), heminephrectomy (21), 
nephroureterectomy (20), ureteral reimplantation (15), 
and other (14) (Figure la). The surgical approach was 
transperitoneal in all cases, with 7 cases also involving 
transvesical access for cross-trigonal Cohen technique 
ureteral reimplantation. The average patient age was 7.3 
years (ränge 1.2 months - 19.1 years). The smallest 
patient in the series weighed 4.1 kilograms. Overall 
conversion rate was 5.7% (10/175), with the procedure 
being successfully completed laparoscopically in two of 
these circumstances. Reasons for conversion were due 
to unexpected issues arising with equipment or theatre 
staff (5/10), inadequate visualisation (3/10), lack of 
operative domain (1) and technical difficulty (1). 



Fig. 1 a) Distribution of caseload by procedure, b) typical 
modified flank patient positioning and port configuration for 
robot-assisted pyeloplasty and c) left-sided transperitoneal 
view of a dismembered pelviureteric junction at 
commencement of ureteral anastomosis 
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One intra-operative complication occurred, involving 
estimated blood loss (EBL) of 200mL during 
heminephroureterectomy in a 12-year-old child for a 
non-functioning and refiuxing left lower moiety. This 
event related to parenchymal surface bleeding and did 
not impact intra-operative or post-operative care (1.1 
g/dL drop in post-operative haemoglobin concentration 
therefore EBL suspected to have been over-estimated). 
Overall 30-day post-operative complication rate was 
5.9% (Table 1). Operating times were variable 
depending on the relevant procedure, ranging from 47 - 
367 minutes for total operating time, and 6 - 280 
minutes for console time. Median total and console 
operating times for pyeloplasty were 226 minutes (ränge 
153 - 340) and 167 minutes (ränge 70 - 280) 
respectively. Median length of post-operative hospital 
stay was 24 hours (ränge 3-144 hours). Two patients 
(2.3%) required re-do robot-assisted pyeloplasty. 

DISCUSSION 

The application of robot-assisted surgery in paediatric 
urology is diverse, reflective of the wide ränge of 
pathology and patient size encountered in this specialty. 
There is a dominance of reconstructive procedures. The 
most eligible application seems to be pyeloplasty. 
Smaller instruments more appropriate for paediatric 
patients are required, which would likely facilitate use 
of current robotic Systems in younger patients, and be 
better suited for procedures in particularly confmed 
anatomical locations such as intra-vesical ureteral 
reimplantation and retroperitoneal pyeloplasty. 
Currently available 5mm instruments are less versatile 
compared to 8mm instruments due to limited catalogue 
ränge and longer distal articulation length compared to 
the compact 8mm Endo-Wrist® design instruments. 
Conversion and complications rates in this series seem 
low and favourable in the context of conventional 
laparoscopy and open technique surgery. Technical 
advantages of robot-assistance, such as enhanced 
dexterity, precision, visualisation and ergonomics, 
might enable minimally invasive approaches to complex 
reconstructive urological surgery in children that 
otherwise would be beyond the remit of conventional 
laparoscopy. 


Table 1 Post-operative complications occurring in the study 
cohort (n = 175) graded according to the Clavien-Dindo 
Classification 


Clavien 

Grade 

n 

Description 

I 

3 

Post-operative ileus (1), surgical 

emphysema (1), hypertrophic umbilicus 
scar formation (1). 

II 

2 

Post-operative nausea and vomiting (1), 
urinary sepsis (1). 

lila 

2 

Urinary retention requiring urethral 
catheterisation (1), stent blockage 
requiring saline flush (1). 

Illb 

4 

Stent displacement requiring replacement 
(1), recurrent PUJO presenting as acute 
hydronephrosis requiring urinary 

diversion and later re-do pyeloplasty (2), 
acute hydronephrosis due to clot 
obstruction prompting return to operative 
management (laparoscopy) (1). 

IVa 

0 

- 

IVb 

0 

- 

V 

0 

- 
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INTRODUCTION 

Virtual reality (VR) temporal bone Simulation provides 
an interactive three-dimensional recreation of the 
surgical environment. The intended surgical procedure 
may first be performed in a virtual environment using 
case-specific CT data uploaded onto the Simulator. 

The role of VR-simulated case-specific rehearsal in 
carotid endovascular surgery has been reported 1 . Results 
suggest that it facilitates case selection and improves 
surgical performance. 

The Voxelman Temposurg has been validated for 
postgraduate training at Imperial College London . The 
process of uploading data onto a temporal bone 
Simulator for case rehearsal has not been reported. The 
objectives of this study were to determine the feasibility 
of performing case rehearsal and to identify potential 
clinical applications in temporal bone surgery. 

MATERIALS AND METHODS 

The Voxelman TempoSurg was used. Sixteen 
participants were recruited comprising of 8 experienced 
Otologists (minimum of 400 mastoid operations as 
primary surgeon) and 8 Otolaryngology trainees (mean 
experience of 19 months). All trainees had prior 
experience in temporal bone drilling. 

Sixteen formalin-fixed cadaver temporal bones were 
scanned using Philips iCT 256 CT-scanner. Data was 
saved to a CD-ROM for upload using the data import 
module. Data transfer and retrieval were evaluated for 
slice spacing and thickness, field of view and intensity 
ränge. The time for each temporal bone upload was 
recorded. 

Participants were assigned a cadaver temporal bone and 
its corresponding upload. All subjects undertook a 90- 
minute temporal bone dissection on the Simulator before 
performing 3 dissection tasks on the case upload and 
cadaver temporal bones: extended cortical 


mastoidectomy, posterior tympanotomy and 
cochleostomy. Standardised written instructions for 
each task were provided. Successful task completion 
was judged by 2 co-authors using a task-based checklist. 

The role of case rehearsal was evaluated across 4 
domains and the accuracy of anatomical representation 
for 6 temporal bone structures recorded. Questions were 
rated on a 5 -point Likert scale, in which 1 represented 
strongly disagree and 5 as strongly agree. A score of 4 
was the minimum threshold for acceptability. 
Differences between the experienced and trainee group 
were analyzed with the independent t-test. A p value of 
<0.05 was considered significant. 

Qualitative data was collected using Videos of 90- 
minute focus group sessions where both groups were 
asked open-ended questions. The Videos were saved as 
digital flies, manually transcribed and underwent 
thematic analysis 

RESULTS 

Each data set was approximately 156 slices, 
0.33mm><0.33mm pixel size and 1mm slice spacing. All 
temporal bones were devoid of pathology. The entire 
mastoid process was inadvertently not captured in 2 
temporal bone scans. There was artefact in 3 temporal 
bone uploads. This was due to image interference from 
the attached cadaver identity tags. 

Optimal 3D reconstructions were created using a 
Segmentation threshold value of 450 Hounsfied units to 
defme the ossicular chain. 
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DISCUSSION 



Fig. 1 Suboptimal temporal bone VR upload due to artefact 
(A) 

User activity associated with the Segmentation process 
accounted for the majority of the upload time. The time 
for each upload decreased with experience (mean time: 
20.7 minutes, ränge 10-40 minutes, R 2 =0.9, p=0.0001). 


Table 1 Role of case rehearsal in temporal bone surgery 


Role of case 

rehearsal 

Total Cohort 

(n=16) 

% rated ! Mean 

i 

i 

Likert j 

>4 i 

i 

Trainer 

( n= 8) 

Mean 

Trainees 

(n=8) 

Mean p 

Confidence 

75 j 4.1 

3.6 

4.5 

0.06 

Pre-op 

i 

i 

i 





75 ! 4.1 

3.6 

4.5 

0.1 

planning 

i 

i 

i 




Training 

94 | 4.3 

3.9 

4.6 

0.04 

Overall role 

81 | 3.8 

i 

i 

3.6 

4 

0.4 


The role of rehearsal across 4 domains is shown in 
Table 1. Its role for training scored highly (94%). There 
was a significant difference in the mean score between 
the trainer and trainees for this domain only (p=0.04). 
Thematic analysis demonstrated perceived advantages 
and limitations. The ossicular chain was the only 
anatomical structure positively rated by the experts. 
Dura was rated suboptimal; the most frequently reported 
limitation being a lack of auditory cues whilst drilling 
over the tegmen. Participants reported that areas of 
anatomical Variation conveyed to the user in the virtual 
setting influenced subsequent task performance on the 
cadaver model, including degree of pneumatisation, low 
dura and high sigmoid sinus (n=4). 

The following procedures were identified as potential 
applications: ossicular chain surgery, paediatric cochlear 
implantation and congenital bony anomalies. Rehearsal 
of procedures involving the facial nerve and removal of 
cholesteatoma were not feasible on the existing platform 
due to “ lack of depth perception during deeper 
temporal bone dissection 


The study demonstrates a streamlined process to enable 
this application to become a useful tool for the 
practising otologist. The results support case rehearsal 
for confidence, planning and training. It allows the 
intended procedure to be rehearsed; improving technical 
skills and case familiarisation were rated highly by the 
trainees. Both groups strongly attributed a role to 
improve the understanding of anatomy and facilitating 
surgical planning. Qualitative data suggests that 
successful completion of a task in the virtual setting was 
able to predict success of the same task in the cadaver. 

Representation of the ossicular chain was positively 
rated. Correspondingly, procedures involving the latter 
represented a potential application. There was 
consensus regarding lack of a role for rehearsal in facial 
nerve decompression due to suboptimal soft tissue 
reconstruction and poor depth perception when drilling 
deeper structures. These Undings suggest that high 
levels of realism are necessary to enable clinical 
application and that a low level of realism is predictive 
of lack of clinical application. In this evaluation pixel 
size was 0.33mmx0.33mm with 1mm slice spacing. 
Non-isotropic image data could potentially alter 
accuracy. 

Although experienced surgeons are not necessarily 
experts in surgical education, their opinion may provide 
a more meaningful barometer of the efficacy of 
Simulation for case rehearsal. In this regard it was 
suboptimal. It is not surprising that the trainee group 
rated the System highly for technical improvement. 
Their demographics demonstrate reasonable experience 
which provides a basis for comparison with other 
methods for training. This group may find simulated 
rehearsal more useful than experts due to greater 
familiarity with Computer based technology and a better 
appreciation of their own learning needs. 

Case rehearsal should be viewed as a potentially useful 
adjunct to the existing methods of pre-operative patient 
evaluation. Further clinical study is necessary to 
evaluate whether Simulation can improve the surgeon’s 
perceived and actual performance and, ultimately, 
patient safety. 
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INTRODUCTION 

Düring the last decade, there has been a significant 
investment in acquiring robot-assisted surgical Systems 
in the Gulf Cooperation Council [1,2]. Nevertheless, the 
robot-assisted surgery (RAS) did not show a high-level 
of activity as explained by several key region- specific 
factors [2]. Qatar has witnessed a steady increase of 
RAS activity over the past five years. In this paper, we 
report details about RAS performed in Qatar and 
provide some insights on the obstacles hindering growth 
and factors that will help to overcome them. 

ROBOT-ASSISTED SURGERY IN QATAR 

Since the first acquisition of daVinci® robots in 2008, 
183 cases of RAS have been documented in Qatar. 
Urology represents most of these cases (69%) and is 
followed by cardiology (22%) as shown in Fig. 1. The 
importance of urology in RAS is the result of a constant 
and substantial growth with a yearly increase of cases 
ranging from +30% to +100%. 

This increasing number of cases provided more 
experience to surgeons, increased their efficiency, and 
reduced the time spent in the operating room. Fig. 2 
illustrates the latter outcome for the three most common 
types of urological RAS: pyeloplasty, prostatectomy, 
and partial nephrectomy. 

In terms of patient outcome, their corresponding 
post-operative stay at the hospital has been shortened in 
average by 22% with RAS (most significant reduction 
for pyeloplasty with 55% reduction, as shown in Fig. 3). 
And complications with more than two weeks post- 
operative stay have the same proportion in RAS than in 
open surgeries (respectively 5.6% and 6.8%). 

GROWTH OBSTACLES 

Even though RAS has witnessed substantial growth 
since 2008, it has been limited by several factors: 



Robot-Assisted Surgeries in Urology 



Adrenalectomy 


Nephrectomy 



Fig. 1 [Up] Repartition of robot-assisted surgeries (RAS) in 
Qatar over the 183 performed since 2008. [Bottom] 
Repartition of robot-assisted surgeries (RAS) in urology. 


It takes time for patients, surgeons, and anesthetists 
to understand, accept, and finally trust RAS as an 
alternative treatment replacing number of 
traditional interventions. 


An open surgery team was established side-by-side 
with the RAS initiative. The number of cases was 
split between the teams in favor of open surgery as 
RAS was limited by availability of mentors. 

The Hamad Medical Corporation (HMC), which is 
the only RAS provider in Qatar, often experience 
over-demand for beds and operating rooms. 


RAS was only devoted to selected cases in urology 
for which patient’ s benefits were clearly identified 
(mostly in case of cancer and reconstructive cases). 

Population in Qatar is relatively young (median age 
of 29.9 years in 2011) due to a significant expat 
labor force [3]. The number of RAS candidates is 
thus limited. 
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FUTURE GROWTH FACTORS 

By understanding these limitations, we believe that the 
following factors will accelerate RAS growth in the 
next coming years: 

+ RAS will expand to include more specialties 
( e.g . gynecology and bariatric). 

+ HMC will increase the capacity of its hospitals. 

+ RAS is gaining wider acceptance partly due to the 
establishment of the Qatar Robotic Surgery Centre 
(QRSC), a local training and research center in 
surgical technologies that further promotes RAS in 
Qatar. 

+ HMC is taking the initiative to attract more 

international clinical experts and mentors by 
participating and organizing RAS-related clinical 
Conferences and through international 
collaborations on research projects via QRSC. 

CONCLUSION 

We have listed many factors that can accelerate the 
growth of RAS in Qatar. Nevertheless, we did not 
mention an important growth factor not only applicable 
to Qatar but also worldwide: the development of novel 
surgical robot technologies that can have a substantial 
impact by improving benefits of RAS over more 
traditional surgeries and/or expanding the application of 
RAS to other surgeries. 

Whether through the clinical RAS programs at HMC 
or the research and training projects at QRSC, Qatar has 
grown its expertise and is preparing the ground to be the 
regional hub for clinical and technological innovation in 
surgery and related fields. Indeed, recent national 
strategies formulated for health [4] and research [5] 
confirm the national will to achieve this ambitious goal. 
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Fig. 2 Evolution of surgery duration between 2008 and 2012 
for most common urological interventions: partial 
nephrectomy, prostatectomy and pyeloplasty (resp. 21%, 24%, 
and 30% of urology cases). 
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Fig. 3 Relative difference of post-operative stay between RAS 
and open surgery for most common urological interventions 
from 2008 to 2012: partial nephrectomy, prostatectomy and 
pyeloplasty (resp. 21%, 24%, and 30% of urology cases). 
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INTRODUCTION 

The central dogma of keyhole neurosurgery is the 
minimisation of intraparenchymal dissection and 
reduction of exposure and retraction of unaffected brain 
tissue in order to reduce approach related morbidity. 
The next generation of progress in this field would seek 
to further exploit the natural anatomical corridors and 
cerebrospinal fluid (CSF) pathways within the brain. 
The latter have already been extensively used for 
conventional microneuro surgery. The challenge lies in 
the difficulty of triangulating rigid tube-shaft 
instruments and performing fine dissection of delicate 
neurovascular structures through a narrow working 
channel. The construction of robotic platforms with 
articulated instruments may allow improved 
manipulation through keyhole approaches. A necessary 
prerequisite for these platforms is knowledge of the 
microsurgical anatomy of these working spaces. 

The aim of the present study was to anatomically defme 
and measure the intracranial cisterns utilised during 
keyhole neurosurgical approaches in a high-resolution 
Magnetic Resonance Imaging (MRI) brain atlas. 

MATERIALS AND METHODS 

A literature search was performed to defme the working 
spaces of the suprasellar, quadrigeminal and 
cerebellopontine cisterns [1]. The SRI24 multichannel 
atlas of normal human brain structure was used as a 
reference System [2]. The SRI24 atlas was recently 
developed using template-free registration of 3T MRI 
images of 24 normal control volunteers (12 male and 12 
female; 12 young and 12 elderly), and is unique in being 
well suited to both spatial normalization and label 
propagation (atlas-based Segmentation). 



Fig. 1 SRI24 Axial Tl-weighted MRI illustrating the 
suprasellar cistern bordered by gyrus rectus of the frontal 
lobes anteriorly, uncus of the temproal lobes laterally, cerebral 
peduncles posterolaterally, and the interpeduncular cistern 
posteriorly. 


Atlas flies were converted to DICOM using MedCon 
v0.12.0. Two independent observers (HJM and AHH) 
measured distances on Tl-weighted MRI using the 
OsiriX Imaging Software v5.6 (see Fig. 1), and 
segmented and measured volumes using itk-SNAP 
v2.4.0 [3]. Mean values for distances and volumes were 
calculated, and the concordance correlation coefficient 
between the two observers determined. 

RESULTS 

The SRI24 atlas was crisp enough for region defmition, 
allowing precise atlas-based Segmentation. Distances 
and volumes are summarised in Table 1. 


Table. 1 Dimensions of the suprasellar, quadrigeminal and 
cerebellopontine cisterns. A-P = Anterior-Posterior; S-I = 
Superior-Inferior; Lat = Lateral; Vol = Volume. 


Cistern 

A-P 

S-I 

Lat 

Vol 

Suprasellar 

25mm 

18mm 

20mm 

6525mm 3 

Quadrigeminal 

17mm 

16mm 

20mm 

2953mm 3 

Cerebellopontine 

20mm 

19mm 

15mm 

3428mm 3 


The concordance correlation coefficient was: p 
(precision) = 0.97, C b (accuracy) = 0.99, and p c = 0.96. 


DISCUSSION 

In this study we have successfully extracted information 
on the deep cisterns that are utilised in keyhole 
neurosurgical approaches using a novel MRI atlas that 
incorporates data from multiple volunteers while 
maintaining salient anatomical features. The suprasellar 
cistern was the largest of those assessed; suggesting that 
the keyhole supraorbital subfrontal approach may be 
most suitable as an index procedure for putative robotic 
platforms. It is likely that during actual operations the 
working spaces encountered will be greater as CSF is 
drained and instruments dynamically retract tissue. 
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INTRODUCTION 

The mitral valve is one of the four valves of the human 
heart. Serving as a passive check valve, a healthy mitral 
valve permits the flow of oxygenated blood from the 
left atrium to the left ventricle while preventing 
backflow. This is accomplished via the mutual collision 
of the leaflets and the tethering forces of the chordae 
tendineae (Figure 1). When backflow occurs, the 
preferred method of treatment is mitral valve repair. 

Mitral valve repair is a technically challenging 
procedure in which the surgeon modifies the native 
valve tissue through a series of tissue resections and 
suturing. While the goal is to restore valve function, i.e., 
ensure proper valve closure under systolic load, doing 
so is difficult as the heart is arrested during the 
Operation. As a result, rather than being directly 
observed, closed valve shape must be predicted through 
analysis of the flaccid valve and the surgeon ’s past 
experience. 

In analyzing the valve, the goal is to identify regions of 
abnormal leaflet mobility. This is achieved by pulling 
the chordal insertions upward using a nerve hook. 
Regions of excessive or restricted leaflet motion are 
recorded [1]. With this information, the surgeon can 
then visualize and predict the shape of the loaded valve. 
In a pathological valve, the prediction is used to develop 
a repair plan to restore valve function. Therefore, not 
only must a surgeon be capable of implementing a 
desired set of valve modifications, he or she must also 
be able to analyze a valve intraoperatively to predict 
closed valve shape. 

This work presents the first steps towards the 
development of a computer-based training platform for 
mitral valve repair. In it, we demonstrate it is feasible to 
construct a platform for assessing a user’s ability to 
predict the closed valve shape of a flaccid valve. 

MATERIALS AND METHODS 

Simulation : In prior work, we have developed and 
validated an interactive computer-based surgical 
Simulator for valve repair [2]. The System accepts a 
subject- specific valve image as input and permits user 
interaction using a haptic input de vice (PHANTOM 
Omni). This allows for the valve to be rendered both 
visually and haptically. However, the use of a haptic 
device necessitates a 1 kHz update rate to ensure 
stability and haptic transparency. To produce such a 
fast Simulation, a mass-spring approximation of a finite 



Fig.l The mitral valve is a key cardiac structure that regulates 
the flow of oxygenated blood. (Image source: Patrick J. Lynch, 
medical illustrator, C. Carl Jaffe, MD, cardiologist). 

element model is implemented as mass-spring models 
are more computationally efficient than their finite 
element counterparts with a minimal change in accuracy 
[3]. Model parameters such as leaflet stiffness are taken 
from the results of ex-vivo mechanical testing found in 
literature. 

Using the haptic Stylus, not only can the user feel the 
leaflet tissue, but can also pull on a desired chordal- 
leaflet insertion point. To do so, a stiff spring-damper is 
engaged between the Cursor and insertion point allowing 
for the stable bilateral application of forces to the user 
and the mesh (Figure 2) [4]. Prior to being rendered to 
the user, forces are scaled and transformed to align with 
the user’s viewpoint. 

Evaluation: This platform was used to assess the ability 
of subjects, namely 3 ld year medical students and 
cardiac surgeons, to predict closed valve shape from an 
unpressurized valve. Subjects were first presented with 
visual-only renderings of four closed valves (Figure 3) 
with valve closure induced using the Simulation 
platform prior to being presented to the user. Variations 
in closed valve shape resulted from differences in chord 
lengths and papillary locations (also generated with the 



Fig. 2 Subjects use the surgical Simulator to analyze the 
mobility of leaflet segments. The blue sphere serves as the 
subjects’ Cursor. 
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Fig.3 For each trial, subjects were presented with four choices 
of closed valve geometry and asked to select which one 
corresponded to the open valve presented. Note the differing 
locations of regurgitant orifces and/or prolapsing segments. 

Simulator). In an attempt to isolate the subjects’ ability 
to predict closed valve shape, accompanying lesions 
typically associated with these conditions, such as 
changes in leaflet and annular geometry, were 
intentionally excluded. This was done to prevent the 
surgeons from integrating advanced pathology-specific 
knowledge that medical students might not possess. 

After examining the four choices from all angles, the 
subjects were presented with an atrial view, that which 
is seen intraoperatively, of a single open, unpressurized 
valve on the Simulation platform. They were allotted a 
maximum of 3 minutes to pull on various chordal 
insertion points using the haptic device with the goal of 
identifying regions of restricted and excessive leaflet 
motion. Upon completion of valve analysis, the subjects 
were again presented with the four closed valve choices 
and asked to select the one that he or she believed 
corresponded to the open valve presented. Answers and 
time elapsed during valve analysis were recorded. This 
process was repeated for a total of 10 trials per subject. 

RESULTS 

Nine subjects were tested: 6 medical students and 3 
cardiac surgeons. The average number of correct 
responses among the medical students and surgeons was 
2.5 and 6.3 respectively (Figure 4). Surgeons performed 
better at a statistically significant level (p=0.012). 

Time to perform valve analysis also varied between 
surgeons and medical students. While surgeons always 
used the full allotment of time (180s), medical students 
fmished in less time (p<0.001), averaging only 90s per 
trial. This increase in speed was trial-dependent with 
later trials requiring less time to complete. 



Fig. 4 Mean performance of the subject populations is shown 
via the bar graph with individual subjects’ scores overlayed. 


DISCUSSION 

Despite a small sample size, the difference in accuracy 
between the two populations was statistically 
significant. This demonstrates not only the difficulty of 
the task, but also the importance of training. Even after 
three years of study, including coursework on cardiac 
pathophysiology, the medical students’ responses were 
equivalent to random guessing. While this can be 
partially attributed to a less thorough understanding of 
mitral valve physiology, the different manner in which 
the two groups analyzed the valve was notable. 

The cardiac surgeons were significantly more 
systematic in their analysis. The process was highly 
repeatable and consistent, always starting in the same 
region of the valve and progressing identically with 
variations occurring only after the entire valve was 
inspected. While this provided a consistent baseline for 
comparison, it also was more time-consuming 
explaining the significant difference in elapsed time. 
Conversely, the medical students used a more ad-hoc 
approach, pulling on all regions of the valve but in 
inconsistent Orders and directions. Therefore, it was not 
surprising to see the best scores achieved by the cardiac 
surgeons with the top score achieved by the most 
experienced surgeon. All of this occurred despite the 
absence of traditional accompanying lesions and a 
preoperative diagnosis that cardiac surgeons typically 
receive. 

In this work, we have demonstrated that our Simulation 
platform is capable of assessing a subject’ s ability to 
analyze a mitral valve and predict closed valve shape. 
Given the demonstrated value of formalized training in 
valve analysis, we aim to couple this assessment 
technology with automated, personalized instruction to 
improve and accelerate the education of future cardiac 
surgeons. 
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INTRODUCTION 

Confocal endomicroscopy is a novel imaging tool that 
allows real-time morphological imaging at cellular and 
subcellular resolution. Over the last few years, its ability 
to differentiate characteristic morphological features of 
neoplastic and non-neoplastic changes of the 
gastrointestinal tract [1], biliary tree [1] and lung [2] has 
been increasingly recognised. However, its role outwith 
the endoluminal environment for surgical applications 
has been comparatively sparsely investigated. In 
particular, there is little reported on its ability to 
characterise morphological features beyond mucosal or 
luminal epithelial architecture. The aim of this 
feasibility study is to test the hypothesis that confocal 
endomicroscopy could be used to visualise soft tissue 
morphology in an intraoperative field. 

MATERIALS AND METHODS 

Under the study protocol approved by the Home Office 
(No. 80/2297), a neck dissection was performed on an 
anaesthetised six-month-old white Landrace crossbreed 
female pig (70kg). A vertical midiine incision was 
performed and the strap muscles, thymus, thyroid, 
carotid artery, internal jugular vein, trachea and 
esophagus carefully dissected (Figure 1) and the 
respective vascular pedicle / vasculature of thyroid and 
thymus preserved. Five mls of intravenous sodium 
fluorescein 10% (Martindale’s, UK) was administered 
as tissue contrast and the confocal endomicroscopy 
miniprobe (2.6-mm diameter Ultra-Mini-0 ™) (Mauna 
Kea Technologies, France) was deployed in-situ. The 
miniprobe delivers a 488-nm-wavelength excitation 
laser light perpendicular to the surface of tissue at a 
confocal depth plane of 60-pm from the surface with an 
optical sectioning of 10-pm. The resulting emissions 
between 500- and 650-nm from fluorescing tissues were 
collected at 12 frames per second. Still confocal images 
were obtained with a field of view of 240-pm and lateral 
resolution of 1.4-pm. Real-time image mosaics 
(stitching of adjacent images next to each other) were 
created using Cellvizio, an image processing Software. 

RESULTS 

Within several minutes of intravenous administration of 
sodium fluorescein, the cellular boundaries of fat cells 
were easily identifiable as polygonal- shaped dark- 
coloured structures with thin hyperfluorescent borders. 
The longitudinal muscle Tibers of both skeletal (strap 


muscles of the neck) and longitudinal muscles of the 
esophagus were visualised as long cylindrical structures 
separated by hyperfluorescent connective tissue with 
numerous cross -striations visible on skeletal muscles. 
The thyroid gland parenchyma had a distinctive 
appearance, with round- or ovoid- shaped follicular cells 
and a thin layer of hyperfluorescent secretory 
epithelium lining a central lumen filled with colloid. 
Within the cortex of the thymus gland, there were 
numerous vascular fibrous septae in between thymic 
lobules (Figure 2a-g). We were unable to visualise 
individual lymphocytes within the cortex of the thymus 
gland. There was no distinctive morphological 
architecture visualised on the tunica adventitia of major 
vessels of the neck. 

DISCUSSION AND CONCLUSION 

It is evident that morphological architecture of most soft 
tissues within the intraoperative field of the neck could 
be visualised in real time using confocal 
endomicroscopy, hence suggesting that image 
acquisition of tissue morphology is not just limited to 
mucosal or epithelial linings of the gastrointestinal tract 
or lung, respectively. The clinical value of confocal 
endomicroscopy in neck dissection is likely to lie in its 
ability to identify parathyroid tissues during 
parathyroidectomy, and to differentiate malignant from 
unaffected lymph nodes. To this end, further work is 
planned to ascertain the morphological features of 
lymph nodes and parathyroid tissues. Whilst the 
flexibility of the miniaturised confocal miniprobe could 
be an advantage for deployment during minimal access 
neck surgery, one of its key ergonomic challenges of a 
hand-held probe is the surgeons’ ability to consistently 
obtain high quality images in the presence of tissue 
deformation and hand tremor. This might warrant the 
use of a mechatronically enhanced or robotic assisted 
delivery platform to achieve slow and controlled 
actuated movements with precision. 
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Fig. 1 Intraoperative images of anatomical neck dissection in a porcine model. (a) Thyroid gland with a confocal endomicroscopy 
miniprobe deployed perpendicular to its surface, thymus (black arrow), trachea (yellow arrow) and strap muscles (arrowhead); (b) 
With the thymus retracted, the internal jugular vein (black arrow), carotid artery (yellow arrow) and esophagus (arrowhead) was 
easily visualised. 



Fig. 2 Real-time confocal endomicroscopy mosaics of soft tissues (a) Polygonal shaped fat cells; (b) cylindrical shaped longitudinal 
muscle cells of the esophagus; (c) striated cells of strap muscles; (d) Follicular cells of the thyroid gland using topical application of 
0.05% acriflavine hydrochloride as contrast agent on cadaveric porcine models; (e) In vivo thyroid follicular cells; (f) Opaque 
appearance of colloid in the central lumen of thyroid follicular cells; (g) Thymus lobule with a surrounding vascular septa. 
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INTRODUCTION 

Single-port laparoscopy (SPL) is currently recognized 
as a revolutionary surgical technique that exploits a 
single incision (most commonly at the umbilicus level) 
for inserting multiple surgical instruments into the 
abdominal cavity, thus guaranteeing a scarless surgical 
Intervention [1]. Despite the improved cosmesis, SPL 
introduces several technical restrictions for the surgeon, 
e.g. limited internal workspace, difficulty of 
triangulation and fatigue [2]. In order to overcome these 
drawbacks, novel instruments and innovative technical 
Solutions are mandatory. However, robotic Systems 
specifically designed for SPL are limited to research 
prototypes. The common thread of such Systems is to 
bring an increased number of Degrees of Freedoms 
(DoFs) intracorporeally, thereby augmenting the 
available workspace and the easy of surgical 
manipulation [3,4]. 

In this framework, a novel teleoperated robotic 
System for SPL has been developed by the authors 
group. SPRINT (Single-Port lapaRoscopy blmaNual 
roboT) robot is a master-slave teleoperated robotic 
platform for bimanual single-access surgical 
interventions [5]. It consists of a bimanual slave 
manipulator, a stereoscopic camera (Karl Storz, 
Tuttlingen, Germany) and a dedicated master console, 
through which the hand movements of the surgeon are 
translated and replicated by the robotic arms. 



Fig. 1 The upgraded Version of the SPRINT System: (Left) 
robotic platform and (Right) the console (developed at the 
Hamlyn Centre for Robotic Surgery by Prof. Guang-Zhong 
Yang). 

Each robotic arm has six active DoFs plus the 
gripper, and by virtue of its mechanical design it 
achieves the workspace necessary for performing 
complex surgical interventions [5]. The joints sequence 


in each arm is a Roll-Pitch-Pitch serial chain (proximal 
joints), followed by a Roll-Pitch-Roll combination 
(distal joints). The proximal joints comprise the 
shoulder and the elbow mechanisms, which are actuated 
by external and on-board motors respectively. As for the 
elbow, the actuation of the distal joints (forearm, wrist, 
and tool rotation) is operated by on-board motors, 
whereas the actuation of the gripper is performed by 
another external motor through a sheath-cable 
mechanism. 

As reported in [6], the dexterity of the robotic arms 
in a master-slave teleoperated configuration has been 
already verified. However, a new session test has been 
carried on for validating and assessing an upgraded 
Version of the aforementioned robotic System. 

From the slave side, the main advancement regarded 
the introduction of a new actuation mechanism of the 
shoulders that allows for the insertion of both the arms 
through an introducer of 34 mm in diameter (Fig. 1 
Left). From the master side, the new master console 
consists of three screens (Fig. 1 Right): the middle one is 
a 3D full-HD passive monitor that displays the images 
coming from the stereoscopic camera, the other two 
provide the surgeon with additional data and settings. 

Two Phantom Omni haptic devices (Sensable 
Technologies Inc., Woburn, MA) and a foot pedal 
complete the console and allow for Controlling the robot 
as described in [7]. 

As anticipated, this paper illustrates the validation of 
a new robotic System as an Upgrade of the SPRINT 
robot reported in [5]. To this purpose, 3 surgeons who 
have never been involved in robotic assistive 
laparoscopy or SPL (herein referred as not experienced 
surgeons), and 16 Student surgeons have been included 
in the study. Besides, the results of this study have been 
compared with the ones obtained in [6] with 
experienced surgeons in robotic surgery. 

MATERIALS AND METHODS 

Keeping in mind that surgery can be summarized in 
three very basic tasks, which are exposure (grasping and 
lifting) dissection and tissue approximation (suturing), 
pick and place exercises and intracorporeal suturing 
tasks in a trainer box assembly were designed for a 
reliable pre-clinical System validation and assessment. 
The first exercise consisted in a peg-transfer setup that 
reproduces the first task of the SAGES Fundamentals of 
Laparoscopy manual skills (FLS) tests [8]. The task 
requires the operator to pick up a peg with one robotic 
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arm, transfer it to the other arm, and then place it to the 
other side of the pegboard respecting the numbers on 
the peg holder (Fig. 1 Left). The exercise was 
considered full-fmished once all pegs were transferred, 
successfully or not, from right to left and vice versa. 
Therefore, the time spent for each peg transfer was 
recorded. The second test required the surgeon to 
perform suturing with intracorporeal knots. In this case, 
the time needed to tie one knot has been recorded for 
comparison purposes. Finally, each subject was asked to 
fill in a questionnaire for the qualitative evaluation of 
the different components/characteristics of the whole 
platform. 


RESULTS 

The mean time taken for each peg transfer is showed in 
Fig. 2a. The trend of the curves indicates that the overall 
System allows for the completion of the tests without 
relevant issues (1 out of 3 peg lost after the fourth peg 
transfer, Fig. 2b). 
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Fig. 2 (a) Peg mean transfer time, (b) percentage of pegs lost. 
Session I relates to the experiments performed in [6]. 

On the other hand, besides the fact that all the users 
have completed the suturing task, a large difference 
between experienced and not experienced surgeons 
emerged. The mean time to perform a surgical knot by 
an experienced surgeon was about 331 s, while a not 
experienced surgeon took about 660 s. This may suggest 
that while a lack of experience with robotic assistive 
laparoscopy can be an advantage for performing simple 
tasks as pick and place (Fig. 2), it becomes a drawback 
in case of challenging tasks such as suturing. In fact, 
during pick and place, experienced surgeons can be 
unconscionably brought to reproduce the same 
movements they are used to, thus resulting in a slightly 
deficient performance compared to not experienced 
surgeons. On the other hand a more complex task such 
as suturing re-establishes the performance hierarchies. 



Besides the session tests described above, one of the 
experienced surgeons that participated in the first test 
session [6] has been asked to perform the test with the 
upgraded Version of the SPRINT platform. The 
measured performance, reported in Table 1, partially 
confirms that large timings obtained for the suturing 
task have to be addressed to the lack of training in 
robotic surgery of not experienced surgeons. 


Table 1 Experienced Surgeon: comparison between test 
session 1 and 2, by using the original SPRINT robot [5] and 
the upgraded Version. 



SPRINT 

SPRINT UP 

Peg Mean Time Transfer (s) 

24.80 

27.18 

Percentage of Lost Pegs (%) 

12.5 

8.3 

Suturing Time (s) 

197 

158 


As regards the qualitative evaluation of the System 
performance, we observed that each subject has a very 
personal judgment of the platform, especially for what 
concern the quality of the 3D viewing. On the other 
hand the majority of the subjects agree in the judgment 
of intuitiveness, actuation by foot, available workspace 
and ergonomics of the console. More precisely the 
overall System has been judged to be comfortable, 
intuitive and with a large enough workspace. Contrary, 
the needle driving grasping force was judged not fully 
satisfactory for an effective suturing task. 

DISCUSSION 

Pre-clinical validation of a new robotic platform for 
SPL has been completed with quite satisfactory results. 
A protocol inspired by FLS tests has been followed: 
pick and place and suturing tests have been completed 
by all the surgeons, thus demonstrating the feasibility of 
the design and the overall usability of the System. 

Experimental in vivo testing on large animals (pigs) 
is planned in future works. 
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INTRODUCTION 

Minimally invasive surgery (MIS) and laparoscopic 
tools have revolutionized surgical intervention by 
allowing surgeons to operate via 5- 10mm tools inserted 
in to 3-4 trocars on the body [1]. The impact has 
shortened patient recovery time but it has increased the 
Operation time due to the lack of dexterity of 
laparoscopic tools. A dexterous robotic tool would help 
overcome these constraints. In addition, advances in 
medical imaging such as high defmition endoscopes and 
interventional MR imaging present an opportunity to 
integrate the various imaging modalities with a robotic 
tool [2]. Anastomosis has been selected as a surgical 
task because it is a challenging yet basic task to perform 
laparoscopically and also present in all specialities. For 
this project, we present early experiences in using an 
endoscope-based image guidance framework to 
automate anastomosis. In particular, the image guidance 
framework is designed for paediatric patients where the 
volume and size constraints are more pronounced. 

MATERIALS AND METHODS 

A robotic System, KidArm, was developed to use image 
guidance as a method to automate laparoscopic 
anastomosis in paediatric patients. Kids Arm consists of 
user interface, image processing unit (IPU), robotic arm 
and anastomosis tool. (Fig. 1) 



Fig.l KidsArm System Design 
The user interface is a dual screen Workstation with 
the top screen capable of showing a 3D depth map and 
stereoscopic 3D images while the bottom is a touch 
screen interface for the surgeon to select suture points. 
The image processing unit is a pair of Basler cameras 
placed side by side (-3cm Separation) that simulate the 
endoscope view and provide input in to image 
processing libraries. The robotic arm is a 6 degree of 
freedom (DOF) Denso VP6 robot. The anastomosis tool 
is a modified Covidien SilS Stitch™ [3] which has 2 


additional joints, roll and pitch, to provide more 
dexterity with an ATI Mini40 6 DOF force-torque 
sensor mounted at the interface between the 
anastomosis tool and Denso VP6. An additional flexible 
biopsy tool has been provided for future work on 
automated knot management. The total DOF in the 
System is 8. For control and integration, robot 
kinematics and Controllers have been designed in 
Simulink and implemented via Quanser QuARC 
interface. 



Fig. 2 Left: Robotic arm, Right: Anastomosis tool 

The general testing procedure involved the 
following: the surgeon draws the area of interest on the 
touch screen interface, the IPU generates a 3D point 
cloud of the target area, the surgeon selects 3-5 suture 
points on the vessel opening (area to be sutured) and 
initiates the procedure on the first point, the IPU 
performs dynamic tracking and provides trajectory 
Information to the robotic tip. 

More precisely, the IPU provides image guidance to 
the robot and tool by using stereo cameras and lighting 
to create the 3D point cloud and track the lumen. For the 
image processing, OpenCV and PCL libraries were used 
for 3D reconstruction, rectification and camera 
calibration. Image feature extraction, tool target tracking 
and deformable lumen tracking algorithms were 
implemented as part of this project. The data flow 
diagram of the IPU are shown in Fig. 3. 
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The 3D point cloud is generated with the semi-global 
block matching function from the OpenCV libraries 
where the IPU generates a depth map based on surface 
features [4]. For testing, a projector applied a set of 
features on to silicone lumen to make them 
representative of tissue. 

A variety of descriptors have been previously studied 
for MIS reconstruction and tracking [5]. A 3D 
based approach was used for deformable vessel tracking. 
The 2D image features are matched between the left and 
right images to compute their 3D location. The matched 
2D features was filtered by the epipolar constraint and 
only the 2D features whose disparity values were within 
the predefmed ränge will pass to the next Step. The rigid 
transformation is solved by 1) matching the 2D features 
(SIFT) between the previous and current images to 
establish the correspondences between the 2D-3D 
features; 2) Using Random Sample 
Consensus(RANSAC) to solve the 6DOFs rigid 
transformation by using the 3D locations of the 2D 
features computed in the previous step. The 3D point 
cloud obtained at each frame was first fused with the left 
camera image features and then fitted with a B-Spline 
surface. From this representation, a coordinate frame is 
computed at each suture point. 

When the user selects a suture point from the User 
Interface, the IPU converts that into a 3D point along the 
surface of the B-spline model. Using multiple user- 
selected suture points, the center of the vessel opening is 
calculated based on the instantaneous surface normal 
from the B-spline model. Both the suture point and the 
location of the vessel opening are sent over to the Robot 
PC, such that the Robotic Controller can insert the suture 
needle into the vessel opening center. 

When the target vessel is deformed, the B-spline 
model accommodates. Using the established geometric 
relationship of each suture point with respect to the 
structure of the vessel, the suture point and vessel 
openings are updated such that the System can react to 
the change in position or form of the target and adjust the 
motion as required. 

The IPU also detects the unique 3D features on the 
Tool target that consists of a square patch of black and 
white features. The target mounting near the tip of the 
Anastomosis Tool gives the true position feedback of the 
Point-of-Resolution (POR) to the Robotic Controller, 
which is tracked by the IPU via the left camera image. 

Together with the position of the suture point(s) and 
the Tool POR position, the IPU provides input to the 
control System for generating the motion command 
required to bring the tip of the Anastomosis Tool to the 
suture point via the vessel opening calculated. 
Registration of the Tool tip and the suture point positions 
onto the camera frame is done automatically. 

RESULTS 

For testing, the image guidance framework was tested 
under 2 scenarios: a ridge-based silicone test pad with a 
4cm long x 2mm gap (side-to-side anastomosis) and a 
pair of silicone tubes (-lern diameter and ~4cm long) 
simulating human lumen (end-to-end anastomosis). 
With the IPU algorithm, the System successfully 


generated 3D point clouds for two scenarios. Fig. 4 is a 
3D point cloud of simulated human lumen. 



Fig. 4 3D Point Cloud of Silicone Lumen 
Using the IPU as part of the Kids Arm System, the 
framework was used to select target points and 
automatically suture the 3-5 points on each test 
scenario. With the first scenario, the IPU enabled 
KidsArm to successfully perform 3 automatic sutures 
(see Fig. 5) to within approximate 2 mm of the target 
points. The second scenario was more challenging as 
the physical joint limits and tool size prevented 
KidsArm from suturing both lumen. More work is 
required to miniaturize the tool to avoid the joint limits. 
However, for a single lumen, a total of 3 sutures were 
completed with similar accuracy. 



Fig. 5 Automated side-to-side suturing 


DISCUSSION 

Based on the results, it is possible to use a pair of 
cameras to simulate Stereo endoscopic video as a means 
to provide image guidance for automated robotic 
anastomosis. The availability of natural features and 
deformable tracking remains a challenge. For the 
silicone samples, this was overcome using artificially 
created features of the vessels. However, this may prove 
to be impractical for in-vivo testing. In terms of 
deformable tracking, the side-to-side suturing scenario 
showed that our proposed IPU can perform in this 
constraint. In the end-to-end suturing, the deformations 
are greater and the vessel placement presents a 
challenge with respect to the robot and tool workspace. 
For future work, we believe that an enlarged workspace 
and field of view will enable image-guided anastomosis 
of tubulär vessels. 
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INTRODUCTION 

Robotic thyroidectomy (RT) is a novel surgical 
technique which uses the daVinci robot through an 
axillary incision. It was pioneered in South Korea where 
small thyroid nodules (<lcm) account for the vast 
majority of cases due to a routine population screening 
Programme. The technique has not been evaluated in a 
UK population. The objective was to compare RT with 
conventional surgery for performing 

hemithyroidectomy. 

METHOD 

An ethically approved prospective case control study 
was conducted (Jan 2010-12). Fifteen patients 
underwent RT and another fifteen had conventional 
surgery. Patients with nodules >7cm, thyroiditis or 
cytology suggesting malignancy were excluded. 
Operative outcome measures included procedure time, 
blood loss and complications. Patient reported outcome 
measures included pain, scar cosmesis, voice handicap, 
swallow dysfunction and quality of life (QoL) using 
validated tools. Mean follow-up was 10 months (ränge: 
3-12 months). 

RESULTS 

There was no difference between the 2 groups in terms 
of mean BMI, age, scar length or follow up timeframe. 
The thyroid nodule was completely excised in all 
patients with no conversions in the RT group. The mean 
size of the lobe excised was significantly larger in the 
conventional group (6.1cm control v 5cm RT p=0.01). 
Laryngoscopy confirmed normal post-operative vocal 
cord function in 29/30 cases. In the RT cohort there was 
1 unilateral recurrent laryngeal nerve paresis and 1 
episode of shoulder dysfunction both resolving by 4 
weeks. One further patient developed a seroma which 
required aspiration. Mean blood loss was comparable 
(40ml RT vs 30ml control). Mean operative time was 
significantly higher in RT (228min vs 85min p=0.01). 
Scar cosmesis was superior in RT from 2 weeks to 12 
months post-op (79% vs 62% p=0.05 and 99% vs 90% 
p=0.02 respectively). There was no difference between 
the groups regarding pain, voice handicap, swallow or 
QoL following surgery. 
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Fig. 1 Scar Cosmesis: 100-0 VAS where 100 represents the 
best outcome score. The RT cohort had statistically 
significantly higher mean scores (represented by triangles) 
from 2 weeks to 1 year (p=0.05, 0.03, 0.01, 0.02). 

DISCUSSION 

Far from being minimally invasive, RT involves 
additional dissection and risk to neurovascular 
structures not typically associated with thyroid surgery. 
The primary advantage is the avoidance of a neck scar. 
This is desirable in selected patients with a tendency for 
hypertrophic scarring. RT is safe, feasible and achieved 
superior cosmesis at the expense of time and cost. One 
patient with pre-existing Osteoarthritis of the shoulder 
joint had dysfunction. Nerve conduction studies 
excluded brachial plexus neuropraxia. Subsequent arm 
Position was modified from that popularised by Chung 
[1]. Improvement in trans-axillary exposure is 
warranted; the treatment rationale in the UK and US 
differs with significantly larger thyroid nodules 
undergoing surgery in a patient population which has a 
significantly larger BMI [2]. Validated training methods 
are essential for safe adoption. A randomised clinical 
study is needed to establish the clinical efficacy 
compared with conventional surgery. 
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INTRODUCTION 

The development of minimally invasive and flexible 
access surgery gives rise to the need of surgical tools 
that can enhance surgeons’ dexterity for manipulating 
tissues in narrow spaces [1]. The design also has to meet 
stringent safety requirements. Consequently, the 
robustness of the surgical tools needs to be optimized, 
taking into account the operating scenario. This work 
focuses on the design of a miniaturized wristed surgical 
grasper, with high robustness and dexterity. 

DESIGN 

This work aims to develop a high-dexterity wristed 
instrument with compact sizes that can be passed 
through the instrument channels of the device described 
in [2]. Another important constraint considered, was the 
use of two hinge-joints design, in order to obtain sharp 
bending (Fig 1) of the tool, which are not often 
guaranteed by the use of other flexible technologies [2]. 
Thus, this wristed surgical tool incorporates 3 Degrees- 
of-Freedom (DoFs) with two pin-joints pivots laying on 
perpendicular planes, and the actuation of the jaws. In 
order to control the tool, 5 tendons are needed, each 
bending is driven by an antagonistic pair of tendons and 
the last tendon is used to close the jaws. A torsional 
spring is used to open the jaws when the tendon is 
released. The wristed grasper developed results to have 
very compact sizes, with a maximum diameter of 
4.8mm and a length of 15mm, considering the distance 
between the first bending axis and the tip of the jaws. 



Fig. 1 Scaled-up (20% bigger) sintered prototype. Straight (a) 
and Bent (b) configuration. 

Starting from the proximal extremity of the tool (Fig. 2), 
a base allocates the holes through which two pins allow 


the core joint to rotate around the vertical axis. The core 
of the instrument is composed of a series of disks, 
together with six rolls, to keep the tendons in the centre 
of rotation, minimizing the coupling effect of the serial- 
tendon architecture. Two pulleys are used to control the 
bending in the horizontal plane of the core element. The 
same idea of small pins and pulleys is used to connect 
the core element with the first jaw. The rotation axis of 
the first jaw is horizontal; therefore the jaw inclination 
in the vertical plane can be controlled through the 
tendons attached to the lateral pulleys. Finally, the 
second jaw is connected to the first one thanks to a 
pulley and a pin. In the jaws two groves are realized to 
allocate the torsional spring for the opening function. 
Each pin joint can rotate in the ränge ±90°, thus the 
maximum workspace of the grasper looks like half a 
three-dimensional ellipsoid, with 15mm height and 
1 1 .3mm base-radius. 



Fig. 2 Exploded view of the CAD model. 

FEM SIMULATION S 

Simulation of four usage scenarios where the grasper 
interacts with tissue is conducted. The material of base, 
core element, pulleys and jaw is stainless martensitic 
Steel, with aging treatment and ultimate tensile strength 
of 2000 MPa. The material to model the pins is spring 
Steel ASTM A228 with ultimate tensile strength of 2300 
MPa and finally the rolls have been chosen to be in 
titanium alloy (Ti-6Al-7Nb) with an ultimate tensile 
strength of 1000 MPa. In spite of the biocompatibility of 
the materials, the prototype (Fig. 1) has been 
manufactured with brass rolls for its malleability. Other 
assumptions that have been taken into account are 
related to the interaction of the tool with the tissues. In 
[4], it is shown that surgeons can apply maximum 10N 
on the tissue using laparoscopic tools. While with more 
focus on different types of tissue it is possible to 
estimate the contact interaction force of the grasper 
according to the softness of the tissue. In example as 
shown in [5], the pressure that the grasper applies on the 
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tissue varies from lOkPa (Colon) to 60 kPa (Liver). 
Consequently, considering the worst-case scenario with 
60kPa, the jaws geometry presented in the paper 
produces a 4.8N force. Consequently, a ION load is 
representative of an intense grasping action; therefore it 
has been uniformly applied to the 6x2mm jaws in the 
simulations. Considering the pulley radius of 2.14mm, 
the tendon needs to be tensioned about 15N. The 
presence of the tendons has been modelled through 
pressure distribution on the elements composing the 
tool, avoiding creating a 3D model. To find the 
equilibrium between forces and reactions, the FEM 
problem has been considered to be stationary, so a linear 
direct solver has been implemented. Tetrahedral 
elements have been used to mesh the model, 
considering contact interactions between the different 
parts. The minimum mesh element size has chosen to be 
0.2mm, with a growing rate of 1.2 between 
neighbouring elements. 

RESULTS 

The Simulation results show that the compact design of 
the instrument is suitable for surgical applications. It 
can be seen from the Simulation results that the places 
where the Von Mises stress is higher are in 
correspondence of the connection between the tendons 
and the pulley s, and of the contact component-pins. 
Also, it is evident that the concentration of stress at the 
base of the joint is due to the absence of larger fillets. 
The grasper has been found to be strong enough to lift a 
load of 500g. A potential limitation could be found in 
the tendons or in the arm onto which the wristed tool is 
mounted. These could result to be less strong than the 
surgical tool limiting the applicable force. Between the 
different scenarios, the last one resulted to be the most 
critical, with a high concentration of stress in the core 
element. This is because the middle part presents a 
grove to allow the passage of the tendon for closing 
function. Since the induced stress has a limited impact 
compared to the materials resistance, the design is safe 
for general surgical use. 

CONCLUSION 

In this paper, a miniaturized wristed surgical tool is 
proposed, which is shown to be compact and dexterous. 
The FEM simulations show that the design is robust to 
be used for minimally invasive surgery. 
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Fig. 3 Scenario 1: grasping with straight joint. A grasping 
force (15N) has been applied to the superior jaw. The 
maximum stress has been calculated to be about IMPa (a). 
Scenario 2: grasping with bent joint. A grasping force (15N) 
has been applied. Both the joints have been bent (90°). The 
maximum stress resulted about 3Mpa (b). Scenario 3: Lifting 
of 500g load. A load (5N) has been applied to the tip of the 
jaws and this generated a maximum stress of 720Mpa (c). 
Scenario 4: 5N later force. A lateral force (5N) has been 
applied to the jaws. This generated the higher stress with a 
maximum peak of 1300MPa (d). 
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INTRODUCTION 


Minimally invasive surgery is increasingly common in 
the treatment of cancer and has facilitated a move 
toward personalised surgery. This has driven clinical 
demand for new instrumentation that provides surgeons 
with improved intra-operative Information, particularly 
to assess and differentiate between types of tissues. 
Current options are scarce and prognosis is typically 
made post operatively. The flow characteristics of bio- 
galvanic current have been proposed as a means to 
distinguish tissue type and health [1, 2]. In this scenario 
tissue acts as a salt bridge between two dissimilar metal 
interfaces. The specific electrode/electrolyte reactions 
determine the potential difference across the cell. 
Characterisation of the salt bridge resistance is possible 
through measurement of the combined System when 
current levels are modulated using extemal resistors. 
The apparent simplicity, relative to active frequency 
based techniques such as bioimpedance spectroscopy [3, 
4], makes bio-galvanic characterisation a promising 
technique for intra-operative tissue characterisation. 
However the validity of assumptions made regarding 
the electrode/tissue interface reactions and mechanical 
contact conditions require further assessment prior to its 
application in clinical tissue characterisation. 

MATERIALS AND METHODS 

Bio-Galvanic Approach: A number of processes are 
undertaken when determining the internal resistance of 
a salt bridge (e.g. biological tissue): (1) establishing the 
interface potentials through wet electrode/tissue contact, 
(2) forming a cell using an extemal resistance, (3) 
varying the level of extemal resistance on the cell and 
measuring the current response, and (4) analysing the 
measured data using a suitable model to determine the 
internal resistance. One additional step that can be taken 
is normalisation to the cell geometry to account for the 
electrode contact area and Separation; results presented 
here are not subjected to this process in order to focus 
on fundamental bio-galvanic characteristics. 
Measurement Studies: Measurements were conducted 
on electronic circuit models, salt Solutions and on fresh 
ex-vivo porcine tissues. Animais used were bred and 
sacrificed in accordance with Home Office regulations 
(Animais [Scientific Procedures] Act 1986). Electrode 
materials used were zinc and copper in all cases with the 
half cell reactions and electrode potentials (vs. a 
Standard hydrogen electrode) proposed by Golberg et al. 
[1] of: 


Zn(s) -> Zn 2+ (aq) + 2e~ E° = 0.76V (SHE) 

2 H + {aq) + 2e~ -> H 2 (g ) E° = 0.00T (SHE) 

Giving the full cell reaction and galvanic potential of: 

Zn(s ) + 2 H + {aq) -> Zn 2+ (aq ) + H 2 (g ) AE° = 0.76V 

Characterisation Method: The polished electrode 
surfaces were contacted with wet tissue and 
subsequently connected in series with an external 
resistance. Custom electronics and control Software 
(Lab VIEW, National Instmments) switched the external 
resistance between discrete values, thereby altering the 
current flow through the cell. Figure 1 schematically 
illustrates the implemented System, where the current is 
determined by the voltage drop, V 2 across the extemal 
resistance, R EX t- The bio-galvanic cell is modelled as an 
ideal power source, producing a voltage equal to the 
open circuit galvanic potential (0 CP ), in series with a 
fixed internal resistance, R INT . Using Ohm’s law the 
measured voltage V 2 can be related to the internal 
resistance as shown in equation 1. 



Fig. 1 Proposed equivalent electrical model of the bio- 
galvanic cell under characterisation. 

v 2 = 7 ^ ^ ; r ext ( 1 ) 

v k ext +h int) 

Where V 2 and R EX t represent the vectors of measured 
voltages and corresponding external resistance values 
respectively. Equation 1 can be solved for OCP and 
Ri nt using the least squares Levenburg-Marquardt 
algorithm [5] and re-presented as a continuous function 
as shown in Figure 2. 

RESULTS 

Tests were conducted to demonstrate the efficacy of the 
electronic control and measurement System when 
combined with the proposed characterisation method. 
Figure 2 shows the raw measurement results and model 
fits from a physical electronic analogue, comprising of a 
power supply and a fixed resistor, representing OCP and 
R int respectively. 
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Fig. 2 Characterisation of electronic circuit models simulating 
varied internal resistances with OCP =0.5 V. Determined fit 
Parameters for 1,10&100 k£2 models were 1009±0.29 £2, 
10.04±0.00 k£2 and 100.2±0.64 k£2 respectively (n=10). OCP 
values of 0.501V were determined in all cases. 

Characterisation of ex vivo porcine liver tissue was 
conducted using the same external resistor set. Figure 3 
shows the results obtained for repeat measurements of 
liver tissue in unstrained and strained cases. Applying a 
significant level of strain (-50%) to the tissue results in 
a shift of the model fit curve, representing a reduction in 
tissue internal resistance. 



Fig. 3 Porcine liver tissue under no-strain (blue) and 50 % 
strain (red). Solid lines indicate mean model fits with Standard 
deviation presented as the shaded region (n=5). 



Fig. 4 A typical current profile during a single characterisation 
of 0.1% NaCl(aq) at 25°C. 

In order to understand how the current varies with time 
during characterisation, measurements were taken using 
a precision potentiostat (Ivium Technologies). This was 
conducted in a model System of 0.1% NaCl solution 
(equivalent in conductivity to liver tissue [6]). Typical 
features of the current profile are illustrated in Figure 4. 

DISCUSSION 

When characterising tissues in the proposed manner, a 
number of significant assumptions are made: (1) the 
established electrode potentials are stable and due only 


to proposed half-cell reactions. (2) Tissue acts as an 
ideal salt bridge with a fixed value of resistance across a 
ränge of currents. (3) The electrode-tissue interface does 
not influence the measured internal resistance. (4) 
Internal resistance is specific to tissue type and health. 

In the case of (1), variability in the order of +/- 100 
mV is seen between electrochemical cells resulting from 
non- Standard conditions. With sufficient data, non- 
standard potentials can be predicted using the Nernst 
equation [7]. The model of equation 1 can account for 
inter-test OCP variations thus mitigating the issue while 
providing an additional discrimination parameter. 

Measurements taken using electrical analogues 
(Figure 2) show the efficacy of the test System and 
model. Highly accurate and repeatable predictions of the 
fixed resistance values are shown. When extended to 
tissue tests (Figure 3), strong agreement is again seen 
over a wide current ränge indicating that assumption (2) 
appears valid. However, adjusting cell conditions e.g. 
applying mechanical strain shows significant influence 
on the determined internal resistance. Figure 4 shows 
that there are time-dependent properties inherent in the 
System. These become prominent shortly after external 
resistance Switching signifying apparent interface 
phenomena; charge/discharge of the electric double 
layer and diffusion of ions to/from the metal surface [7]. 
This indicates that assumption (3) is not valid under the 
cell arrangement tested. Thus the determined R INT 
values may include resistance from the interfaces. 

True assessment of the bio-galvanic technique is not 
possible without further understanding and control of 
the influencing parameters discussed. Variations of the 
electrochemical System, the mechanical conditions and 
the chosen electrical model must be considered. By 
addressing these factors, bio-galvanic measurements 
may show potential for intraoperative tissue assessment. 
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INTRODUCTION 

Recently, laparoscopic surgery has become the gold 
Standard for treating a wide variety of pathologies. 
However the narrow field-of-view (FOV) when 
compared to open surgery necessitates a camera 
assistant to orientate and navigate the laparoscope. The 
importance of good camera handling and navigation is 
being recognized and is now included in the training 
curricula for surgical residents [1]. Despite this, 
assistant fatigue, hand tremor and the close proximity to 
the surgeon cannot be overcome from training. 

To overcome these issues, robotic assisted camera 
control Systems have been commercially proposed such 
as the voice activated Automatic Endoscope Optimal 
Position (AESOP) System from Computer Motion 
Inc.[2], the head-motion controlled Endo Assist [3], and 
the Finger j oystick controlled Solo Assist [4]. However 
these Systems change the dynamics of the Operation. 

The purpose of this paper is to introduce a gaze 
contingent robotic camera control System based on real- 
time gaze gestures. The human visual System is used as 
an information gathering function rather than as a 
control input. It can therefore be difficult to distinguish 
solely from their Point-of-Regard (PoR) whether they 
want to move the camera or scan around the scene. 
Potential Solutions to overcome this are using extemal 
input sources such as buttons or foot-pedals [5] and 
dwell-time on fixed regions of the screen [6] to convey 
panning. Unfortunately, these methods introduce their 
own limitations such as input control clutter and or the 
lack of zoom functionality. 

One area that has not been explored in surgical 
camera control is the use of gaze gestures. Gaze 
gestures use predefmed eye movements to prompt 
multiple camera control modes, e.g. panning and 
zooming. This is the first time gaze gesture recognition 
has been used to control a robotic arm for minimally 
invasive surgery. Gaze gestures have been previously 
used for Human Computer Interaction [7]. In this paper, 
Hidden Markov Models (HMMs) have been used for 
gaze gesture recognition to enable the surgeon to 
experience a hands-free control of the camera System. In 
the proposed System, the important role of the surgeon’ s 
eyes is restored back into the control loop of the camera. 

MATERIALS AND METHODS 

In order to recognise the gaze gestures, HMMs were 
used. HMMs represent stochastic sequences where the 
States rather than being directly observed are associated 


with a probability [8]. In our System, a single HMM 
represents one gaze gesture. Two gaze gestures were 
used for the “pan” and “zoom” control of the camera. 
Both gaze gestures are 3-stroke eye movements. The 
“zoom” gaze gesture is defined by gazing from the 
center of the screen, then to the bottom left corner, back 
to the center, and finally back to the bottom left corner. 
The “pan” gaze gesture is a similar sequence but is 
performed by looking at the bottom right corner instead. 

A total of 300 intentional gaze gestures were used to 
train two HMMs. From a 10-fold cross validation on 
various state parameters it was decided that 6 States was 
a good trade-off between recognition accuracy and 
model complexity. The gaze gesture recognition and 
robot control algorithm is shown in Fig. 1. 


Reuse last two strokes in 



Fig.l Gaze gesture recognition and robotic control algorithm. 


The PoR data is median filtered, and then segmented 
using a velocity threshold saccade detection algorithm. 
The algorithm sequentially extracts 3 saccades and 
encodes it using k-means clustering. The encoded 
sequence is then tested against the respective HMMs, 
for “zoom” and “pan” gaze gestures. The recognized 
gesture is the one with the maximum inference value 
from HMM1 and HMM2, given that it is above the 
inference value threshold of 0.65 and 0.7 respectively. 
Once a gaze gestures is recognized, the PoR is sent to 
the robotic arm to control it, otherwise no input is sent 
to the robot. Düring “pan” mode, the camera will follow 
the user’s PoR. Düring “zoom” mode the camera will 
zoom in if the PoR is above the screen horizon, while it 
will zoom out if the PoR is below the screen horizon. 
The camera is stopped by observing a stop region in the 
center of the screen for 500ms. 

The general control scheme of the robotic camera 
manipulator has been based on a Cartesian impedance 
Controller [9] that computes the command torque r c for 
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each joint according to the position error e in Cartesian 
space and the compensation of the whole robot 
dynamics. The aforementioned error is estimated as the 
difference between a reference pose x d provided by a 
minimum jerk trajectory planner and actual pose 
x retrieved from the robot’ s forward kinematics. The 
trajectory planner is updated on-line in compliance with 

x y moving the 

eye eye J G 

camera where the end user is looking. 

RESULTS 

The experimental platform in Fig. 2 includes a Tobii 
1750 eye tracker, a 10mm zero degree Stortz 
laparoscope, a Storz Tele Pack light box, a Kuka Light 
Weight Robot (LWR) [9] and an upper gastrointestinal 
phantom with simulated lesions inside a box trainer. 
Eleven surgeons participated in the clinical experiment 
using the System during three control modes. 1) Gaze 
gesture control; 2) Pedal activated control, where the 
user activates the camera control with a foot pedal but 
directs the camera with their PoR; 3) Camera assistant 
mode, where a novice assistant Controls the camera for 
the subject. The same assistant took part in all trials. 



Fig. 2 The gaze contingent camera control System (left), the 
“zoom” (upper right) and “pan” (lower right) gaze gestures. 


To assess the new gaze contingent camera control 
System, the following quantitative performance 
measures were used; i) Recognition accuracy of the 
HMM gaze gestures; ii) Camera path length, for ability 
and efficiency; iii) Task completion time, to assess 
usability. Results are represented as medians with 
interquartile ranges in parentheses. Occurrence of false 
positive (i.e. when the user does not perform a gaze 
gesture, but the algorithm triggers a gaze gesture) and 
false negative (i.e. when the user performs a gaze 
gesture, but it is not recognized by the algorithm) gaze 
gestures during the experiment were counted via post- 
hoc observation of the recorded camera- view Videos. 
The HMM recognition accuracy was on average 97.0% 
whilst the average false positive rate was 1 .4% under a 
Statistical cohort of more than 200 gestures. 

Task completion time observed across all three 
modalities showed no significant difference (p=0.690), 
see Fig. 3(a). This indicates that our gaze contingent 
System is as competent as the established use of a 
camera assistant. In contrast, a significantly shorter 
camera path length was observed for the gaze gesture 
modality compared to the assistant (0.896m [0.867] vs. 
Ulm [1.26]); p=0.037) as shown in Fig. 3(b). 
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Fig. 3 (a) Task completion time, and (b) Camera path length 
(m) for all control modes. * Significant p values. 


DISCUSSION 

In this paper, we provided a novel approach to control a 
6 DoF camera through the surgeons’ PoR (i.e. 2 DoF) 
via the use of real-time HMM gaze gestures to pan and 
zoom in the Cartesian space. The ability to consistently 
complete a surgical navigation task has been 
demonstrated with the gaze gesture control scheme. The 
efficiency of the System is shown by the significantly 
shorter camera path length compared to the current 
Standard. Validation of the HMMs showed that HMMs 
are effective in recognizing gaze gestures with mean 
experimental recognition accuracy and false positive 
rate of 97.0% and 1.4% respectively. 
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INTRODUCTION 


The paramount emphasis on safety in surgical robotics 
has driven its incorporation at multiple levels within all 
stages of their development, testing, validation and 
deployment [1]. Nonetheless, such Systems fmally have 
come back full-circle to fundamentally rely solely on 
the surgeon-in-the-loop to ensure safe Operation amidst 
a host of real- world uncertainties and complexities. For 
example, the finite life and slack in the cables of the 
passive-robotic surgical instruments lead to tool- 
positioning inaccuracies, requiring surgeons to 
compensate for this error. We would like to make a case 
for taking advantage of the rieh Information content to 
create automated-assists that can help make robotic 
surgeries smarter and safer. Specifically, in this paper 
we seek to develop and implement real-time video- 
based understanding for improved situational awareness 
and context-based decision support in robotic surgeries. 

Current approaches for surgical safety, guidance and 
training are based on tool motion detection using either 
sensing or image-analysis methods. The former methods 
are either marker-based (limited by bio-compatibility, 
additional Instrumentation and occlusions) [2] or joint 
sensing (using encoders or sensors and suffer 
uncertainties from flexible joints). While, the image- 
analysis methods assume clear conditions and require 
additional information such as accurate tool geometry 
and/or joint sensing [3]. Moreover, surgical tool 
detection and tracking, as it has been considered so far, 
can provide only low level feedback tasks [4] and are 
unsuitable to obtain holistic surgical video 
understanding (relationships between tools and its 
environment). In this work we seek to realize this 
objective by virtue of: (i) automated tracking of multiple 
objects (tools); and (ii) building semantic 
labels/attributes on top of our detection module to 
robustly learn safety critical scenarios and idealistic tool 
motions from the Videos. Although there has been 
limited work on attribute labeling directly from video, 
we believe this will be a critical enabler in robotic 
surgeries towards increasing the level-of-autonomy. 

EXPERIMENTAL SETUP 

Real hysterectomy procedures performed using the da 
Vinci Surgical System- Si (dVSS-Si) by an experienced 
surgeon were recorded. Such uncalibrated monocular 
Videos (totally 16 sequences) were sliced into segments 
(150-300 frames per slice) for manual annotation of two 


different tools (clamping and cutting tool) and two 
attributes (blood stained and tool open or closed 
conditions). The annotated frames (ground truth) were 
ensured to cover variations including tool configurations 
and types, occlusion (by tissues, blood stained and other 
tools), specularities (light intensity, smoke) etc to enable 
robust classifiers. Our combined framework for tool 
detection, tracking and attribute labeling is presented in 
Fig.l. Though this preliminary study is restrictive 
(considers only two tools and two attributes), it can be 
easily extended to multiple tools and attributes. 
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Fig.l Video-based Understanding Workflow 


METHODS 

Tool Tracking: The detectors for each tool types were 
learnt using state -of-the-art algorithms [5] that aided in 
capturing the object shape using Deformable Part 
Models (DPM) (consisting of star-structured pictorial 
structure model). DPM links root of an object to its 
parts using deformable springs and proved to be a 
suitable method for articulated surgical tools. A Latent 
Support Vector Machine (LSVM) classifier is then 
learnt to identify multiple tools by extracting Histogram 
of Oriented Gradients (HOG) from annotated bounding 
boxes as in Fig. 2. The location of the bounding box in 
next frame is approximated by mean flow of all pixels 
using a simple tracker based on dense optical flow. The 
resulting pixel velocities and mean velocity of the 
bounding box can then be used to detect scenarios like 
tool leaving field of view, inappropriate motions etc. 



Fig. 2 Learned HOG Templates for Different Tools 

Attribute Labeling: The accuracy of attribute labeling 
is tested using manually labeled bounding box data to 
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Fig.3 Tool Detection and Tracking Results in Real Surgical Videos [Please view in color] 


obtain Pyramid Histogram of Oriented Gradients 
(PHOG) [6] (captures object shape at various scales). A 
Probabilistic Support Vector Machine (PSVM) 
classifier [7] is leamed to estimate the probability of an 
attribute in each state, for e.g., probability of tool open 
or closed States for each frame. By further applying 
Bayesian inference, transition in semantic attributes is 
filtered to improve the accuracy using the output from a 
specified number of past frames [8] and assuming 
mutual independence [9]. Our framework is also real- 
time capable as it is implemented in a causal manner 
(current value depends on only previous frames). 

RESULTS 

The combined tool detection (based on HOG) and 
tracking framework (using the optical flow) performed 
reasonably well (> 70% accuracy based on Pascal 
measure for a total of 4657 manual tool annotations on 
2400 frames) without any restrictive assumptions on 
type, shape, color, view direction etc. Although the 
detector gives true positives and false positives, the 
tracker is initialized only with those detections that have 
high confidence score. Figure 3 shows successful 
tracking of a single tool with appearance changes and 
multiple surgical tools using only monocular Videos. For 
the semantic labeling, same labeled datasets were 
compared with the classifier results (baseline SVM, 
SVM with filtering and the method from [10]) to 
evaluate their accuracy using 10-fold cross validation. 
The attributes labeling were also compared using RGB 
Histograms (capture color variations of the tool), PHOG 
and both of these together as inputs to our classifiers. 
The results are summarized in Table 1 which indicates 
our method with filtering performed better than the 
current literature [10] . 


Table 1. Accuracy Scores for Different Features using 
Baseline SVM, SVM with Filtering and Method from [10] for 
Attributes: Blood Stained (left) and Tool Open/Closed (right). 



Blood Stained Attribute 

Tool Open/ Closed State 

Feature 

Boseline 

Ours 

[10] 

Baseline 

Ours 

[10] 

RGB 

86 . 41 % 

88 . 38 % 

86 . 73 % 

82 . 05 % 

83 . 17 % 

82 . 91 % 

PHOG 

82 . 56 % 

89 . 31 % 

83 . 34 % 

77 . 50 % 

82 . 28 % 

78 . 20 % 

Both 

88 . 32 % 

92 . 70 % 

88 . 68 % 

85 . 45 % 

87 . 09 % 

84 . 84 % 


DISCUSSION 

This work summarizes our current progress in video- 
understanding of real surgeries for enhanced safety and 
decision support Systems. The multi-tool detection and 
attribute labeling algorithms were implemented and 


validated using our existing infrastructure and found to 
be robust to a host of scenarios and there is scope for 
further improvement. Future work to use multiple 
trackers and probabilistically merge the individual 
detection Outputs to improve Overall accuracy is 
currently underway. Our semantic labeling algorithm 
performed better than the one in current literature and 
showed promise for further refinement with extensive 
benchmark data and cross-validation comprising of 
more varying scenarios and appearance changes. Such a 
cascaded framework will prove to be useful from 
diverse perspectives - surgical guidance, safety, 
tracking and also skill assessment. We plan to combine 
this automated framework with our earlier work on 
surgical skill assessment [11] in near future to 
implement a tightly integrated and automated robotic- 
surgical training workflow. 
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INTRODUCTION 

Surgeons experience sensorimotor distortion whilst 
performing Minimally Invasive Surgery (MIS). Löss of 
depth perception, visuomotor misalignment and the 
effects of visuomotor rotation (VMR) contribute to sub- 
optimal visualisation of the operative field of view 
(FOV). Düring MIS, VMR occurs when the laparoscope 
is unintentionally rotated about its long axis, resulting in 
a mis-match of the percieved and actual FOV. 
Disorienting spatial transformations result in rotational 
misalignment between visual perception and motor 
control [1], degrading performance [1, 2] and 

theoretically increasing cognitive load. Robotic 
telemanipulators, currently in commercial use, stabilise 
the horizon, but are unable to correct rotation when it 
occurs (e.g. Aktormed SoloAssist). The impact of VMR 
on performance and eye behaviour was evaluated in this 
study. Given the predicted change in task difficulty, it 
was hypothesised that VMR would evoke greater 
change in pupil diameter, with greater observed average 
dwell times. 


METHODS 

Eight right-handed laparoscopically naive medical 
students (mean age ± SD = 20.4 ±1.6 years) were 
recruited to perform the fundamentals of laparoscopic 
surgery (FLS) peg transfer task. This task requires 
manipulation of triangulär pegs between laparoscopic 
instruments of either hand and placement onto a pin 
board. The task was performed under normal view and 
with the view rotated by 90° (order randomised). 


NORMAL VIEW 



ROTATED ViEW 


r 




Fig. 1 Tobii 2150C is positioned in place of a Standard 
laparoscopic screen to allow eye-tracking whilst the peg 
transfer task is performed in the normal and rotated views. 


Düring the task, the subjects were eye-tracked with a 
Tobii 2150C remote eye-tracker [Fig. 1]. Following each 
task National Aeronautical Space Agency Task Load 
Index (NASA-TLX) questionnaires were completed. 
Primary outcome measures were pupil and gaze metrics 
(pupil diameter and dwell time), and task performance 
measures (time and validated objective score) [3]. The 
secondary outcome measure was the subjective 
workload (NASA-TLX) score. Statistical analysis was 
performed using IBM SPSS vl9. 

RESULTS 

Group average performance was better under the normal 
view versus the rotated view for task time (mean 138s 
vs. 296s, p<0.001) and objective score (mean 162 vs. - 
16.3, p<0.001). Subjective NASA-TLX scores were 
significantly lower in the normal versus rotated view 
(median 50.7 [11.7] vs. 80.7 [21.8], p=0.012), indicating 
higher workload when rotated. Average dwell times 
were significantly shorter for the normal versus rotated 
view task (190ms [90.0] vs. 300ms [22.0], p=0.047). 
Whilst, average combined pupil diameter was 
significantly larger in the normal vs rotated view task 
(6.50 [1.40] vs 5.59 [0.93], p= 0.02). 

CONCLUSIONS 

VMR results in an evidently poorer technical 
performance compared to normal view laparoscopy in 
task naive subjects. This is associated with a predicted 
subjective increase in workload and dwell time. 
However, the pupil diameter change seen was contrary 
to the hypothesis. It is conceivable that VMR is so 
challenging, that subjects are unable to engage an 
appropriate cognitive strategy, whilst attempting the 
task. Further longitudinal studies are required to 
evaluate the effects of repeated practice under VMR on 
the eye behaviour. VMR has a demonstrable effect on 
performance and eye behaviour, and should be actively 
minimised in any future design of robotic tele- 
manipulators. 

REFERENCES 

[1] Zhang L, Cao CG. Effect of automatic image realignment 
on visuomotor coordination in simulated laparoscopic 
surgery. Appl Ergon. 2012; 43(6):993-1001. 

[2] Cresswell AB et al. Methods for improving performance 
under reverse alignment conditions during endoscopic 
surgery. Surg Endosc. 1999; 13(6):591-4. 

[3] Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, 
Schirmer B, Hoffman K; SAGES FLS Committee. 
Surgery. 2004; 135(l):21-7. 


The Hamlyn Symposium on Medical Robotics (2013) 


109 




Compact Modular System for Teleoperated 
Laparoendoscopic Single Site Surgery 

O.J. Isaac-Lowry, S. Okamoto, P. Berkelman 

Dept. of Mechanical Engineering, University of Hawaii-Manoa 

ojil@hawaii.edu 


INTRODU CTION 

Single-port access (SPA) surgery [1] aims to reduce the 
invasiveness and amount of visible scarring from 
traditional laparoscopic surgery. This alternative 
technique passes multiple instruments through a single 
incision. The navel is the dominant single site as it 
leaves no visible scar. Lack of triangulation resulting 
from a common entry point can be resolved with angled 
or articulated instruments. These instruments are 
developed to provide functionality similar to Standard 
laparoscopic instruments. 

Single-port surgery is a strong candidate for 
teleoperated automation. The improved dexterity, 
accuracy, and positional control effectively address the 
manual difficulties associated with re-triangulated 
instruments and reduced ranges of motion. A number of 
single-site robotic Systems have been developed to meet 
this need. However, there are functional challenges 
associated with each of the design approaches. Larger 
Systems such as the Intuitive Surgical da Vinci System 
[2] equipped with a single port adapter are bulky and 
afford limited angular motion. Smaller designs such as 
the miniature System from Wortman et al [3], the 
Aaraknes System [4], and the internal manipulator 
System of Won-Ho et al [5] suffer from limited 
translation, speed, and force complications. By 
addressing these functional deficiencies, our simple, 
compact, modular teleoperated surgical System provides 
advantages in portability, cost, and ease of use 
compared to existing Systems. Our preliminary design 
as described by Berkelman and Okamoto [6] modifies a 
ViKY XL endoscope manipulator base from 
EndoControl SA and incorporates a set of articulated 
instruments actuated through two independent 
manipulators. The instruments are modular and rapidly 
interchangeable to minimize the need for additional 
incisions. End effector motions are coupled to non- 
zoom endoscope positioning motions for ease of use and 
with instruments removed the assembled robot can be 
autoclaved for simple setup and tear-down. Full 
description of Overall System size, ranges of motion, and 
force generation are described in [6]. 

MATERIALS AND METHODS 

An annotated CAD model of System prototype is shown 
in Fig. 1. Key System elements are listed in [6]. Control 
is achieved with the teleoperation masters, motor 
Controllers, and revised Software from the previous, 

This work has been supported by a grant from Intuitive 
Surgical, NIH grant #5R21EB006073, and the University of 
Hawaii-Manoa Engineering. 



Fig. 2 Interchangable tool tip and articulated wrist shown left. 
Motorized actuation assembly shown right. 

modular University of Hawaii surgical robot System [7]. 
The design of manipulator linkages does create 
individual end effector void geometry as the assembly 
inclination is swept from near horizontal to near 
vertical. However the additional DOF derived from 
assembly base rotation allows positional coverage of 
encountered trajectory voids with minor repositioning. 

To account for single trocar geometry, new modular 
instruments, shown in Fig. 2., were designed to properly 
re-triangulate end effectors in the visible endoscope 
Feld. This is accomplished using an instrument shaft 
with a fixed 45 degree bend and three-dimensional, 
fully- articulated end effector wrists. Wrist structures 
were redesigned to utilize interchangeable tool tips in 
Order to improve System flexibility. Instruments have a 
compact actuator motor footprint to balance weight 
distributions and maximize manipulator ränge of 
motion. Actuator transmission Systems have been 
redesigned to improve serviceability. The number of 
fasteners, components, and joints has been reduced 
through integration of flexible polymer features. Design 
for manufacture (DFM) and assembly (DFA) methods 
were employed to reduce instrument reproduction costs. 
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System prototype and instrument components were 
produced by 3D printing reducing overall development 
lead time. Additionally, total prototype design and 
robustness is enhanced from incorporation of accurate, 
fme-scale custom components. 


RESULTS 

Initial prototype validation has been conducted using a 
single instrument Setup controlled with inverse 
kinematics and velocity joint commands as in Fig. 3. 
Simple surgical dexterity tests were performed to 
evaluate ranges of functional motion, presence and 
shape of unreachable volumes, and limitations due to 
manipulator collisions. The author has been able to 
perform Standard peg transfer motions at rate of up to 
six per minute and verified a minimal rate of single 
manipulator-instrument-support structure collisions. 
Accuracy and repeatability of end effector trajectories 
has been successfully evaluated via optical motion 
tracking using a Northern Digital Polaris Vicra. 
Tracking targets are installed on top of instruments as 
shown in Fig. 3 and measured from a stationary base 
frame to capture all effects from control error and 
manipulator, frame, and mounting hysteresis and 
vibrations. Fig. 4 shows real base frame end-effector 
motion during three consecutive traces of a 30mm 
square using a hand operated Geomagic/Sensable 
Phantom Omni teleoperation master. 

The use of 3D printed manipulator and power 
transmission components has been demonstrated to 
introduce additional friction and hysteresis into System 
motions when compared to machined metal. However, 
these factors have been mitigated through component 
design and post -printing fmishing processes. 



Manipulator 

motors 
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Fig. 3 UH Compact Laparoscopic Surgical Robot shown with 
one modular instrument and motion tracker installed 


DISCUSSION 

Based upon preliminary results, we plan to proceed with 
preclinical trials to evalute two criteria: methods to 
improve control intuition shown by first time operators 
relative to trained medical students and surgeons and the 
total effectiveness of the System when performing 
established surgical tasks. Evaluated tasks are selected 


from the Society of American Gastrointestinal and 
Endoscopic Surgeons Fundamentals of Laparoscopic 
Surgery (SAGES FLS). 

We also aim to capitalize on the rapid 
interchangeability of instruments afforded by a modular 
overall System design. Instrument manipulators will be 
generalized in order to accomodate other, prevalent 
commercial robotic articulated instruments without 
negatively impacting performance. Additionally, we 
intend to continue to explore 3D printing as a 
potentially viable production medium. 
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Fig. 4 End effector motion captured with optical tracking 
during basic robotic dexterity exercise 
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INTRODUCTION 

Magnetic resonance imaging (MRI) is increasingly used 
intraoperatively, because it provides excellent soft tissue 
distinction and on-the-fly adjustment to imaging slice 
Orientation without exposing clinician or patient to 
radiation. For neurosurgical procedures, intraoperative 
MRI holds significant advantages over traditional 
stereotactic methods (e.g. compensation for brain shift), 
and has thereby led to much research toward minimally 
invasive treatments of brain tumors, epilepsy, and other 
neurological disorders. Because the tight confmes of 
MRI Scanners limit clinician access to the patient, an 
MRI-compatible robot is potentially useful for many 
procedures and additionally offers the clinician 
increased accuracy and degrees of freedom within the 
minimally invasive context. 

Affecting 0.5 to 1.0% of the global population, 
epilepsy is a neurological disorder for which surgery has 
been reported superior to prolonged pharmacotherapy, 
yet only a small fraction of potential surgical candidates 
receive treatment [1]. Minimally invasive alternatives to 
conventional resection of seizure foci could side-step 
the potential morbidity and lengthy recovery associated 
with open brain surgery thereby making surgical 
treatment and eure available to a greater number of 
patients with less risk. Interstitial laser ablation of 
seizure foci has been clinically demonstrated using MRI 
to guide ablator placement and MR thermometry for 
thermal dosimetry [2]. This procedure currently requires 
multiple patient transfers between MRI Scanner and 
operating room meaning that introducing an MRI- 
compatible robot could streamline surgical workflow. 
An alternative real-time MRI approach is high intensity 
focused ultrasound (HIFU), yet intracranial targets can 
be difficult to effectively treat due to high reflection and 
absorption of the ultrasound energy by the skull [3]. 

This paper reports a pneumatically-actuated active 
cannula robot designed for MRI-guided ablation of 
epileptogenic foci. An active cannula is a continuum 
robot consisting of concentric, superelastic nitinol tubes 
with pre-curved sections. Complex trajectories are 
realized by translation and rotation of the cannula tubes 
with respect to one another. 

MATERIALS AND METHODS 

Since hippocampal sclerosis is present in a majority of 
epilepsy cases, the hippocampus was selected as the 
target region for thermal ablation, and an occipital 
trajectory was deemed optimal for maximizing coverage 


through a single needle insertion. Folio wing this 
trajectory and performing the ablation can be 
accomplished with five degrees of freedom: two 
translations and one rotation for the active cannula, and 
one additional translation and rotation for the ablator. 
Inclusion of rotation for the ablator provides the Option 
of using an acoustic ablator with radially sectored 
transducers to direct the ablative energy [4]. 

To manipulate the active cannula and ablator, a 
robotic platform was designed and built, as shown in 
Fig. 1. For actuators, pneumatic piston-cylinders with 
fail-safe rod locks were selected instead of piezoelectric 
motors, as Standard piezoelectric motor drives degrade 
image quality unless powered down during imaging [5]. 
Furthermore, pneumatic actuators are quieter than 
piezoelectrics, pose fewer contamination risks than 
hydraulics, and can run off a hospital’s instrument air 
facilities. A detailed mechanical design of the robot is 
reported in [6]. 

A nonlinear, model-based Controller was developed 
for precision position tracking of the actuators [6]. The 
electronics dedicated to Controlling one actuator are: 
one 5-port/4-way proportional spool valve, two pressure 
transducers, one optical encoder for position feedback, 
and one 5-port/2-way valve for rod lock control. Long 
pneumatic tubing (3 meters) and shielded cables 
connect the actuators and optical encoders on the robot 
to the remotely located valves, pressure transducers and 
System electronics. The Controller was implemented in 
MATLAB Simulink with xPC target. A supply pressure 
of 687 kPa was used, providing actuator forces on the 
order of 40 N. 



Fig. 1 Photograph of active cannula robot. Inset shows CAD 
model of cannula and ablator inside hippocampus. 

Three tubes were selected for the active cannula and 
a simulated ablator. Their dimensions are presented in 
Table 1. The active cannula consists of a stiff, straight 
outer tube and inner, pre-curved nitinol tube. The outer 
tube only translates and inner tube both translates and 
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rotates, as shown in Fig. 1 inset. The inner tube was first 
curved by hand, and the resulting radius of curvature 
was estimated using a digital image of the tube. 


Table 1 Dimensions of active cannula and simulated ablator 
tubes 


Tube 1 outer diameter 

3.18 mm 

Tube 1 inner diameter 

2.36 mm 

Tube 2 outer diameter 

1.65 mm 

Tube 2 inner diameter 

1.35 mm 

Tube 2 radius of curvature 

132.2 mm 

Tube 2 are length of curved portion 

52.0 mm 

Simulated ablator outer diameter 

1.18 mm 


The simulated ablator is a superelastic nitinol tube of 
outer diameter 1.18 mm. This is slightly smaller than 
existing MRI-compatible laser ablators [2], but the 
rotational mechanism of the robot readily accepts larger 
tube diameters using an interchangeable collet design. 

Prior to in- Scanner experiments, an initial benchtop 
assessment of cannula tip accuracy was needed. Thus, 
tip position measurements were acquired for 28 robot 
poses using an optical tracking System, the Micron- 
Tracker 3 by Claron Technologies (XB3-BW-H3-60). 
Three optical markers were affixed to the robot front 
plate. Manufactured from acrylic using a laser cutter, 
the plate includes pin-sized holes to assist in placing the 
markers at precisely known positions. Five multi- 
modality fiducial markers were similarly added for 
registration with MRI images. An optical marker was 
fixed to the cannula tip for bench testing with the optical 
tracker; it will be removed for in- Scanner experiments. 

Image registration from optical tracker to robot was 
achieved using point-based registration of the markers 
on the robot’ s front plate. Then, for each robot pose, the 
expected cannula tip position was determined from the 
robot’ s forward kinematics. The resulting model-based 
poses of the robot are illustrated in Fig. 2. Tip error was 
taken as the difference between measured tip position 
and tip position expected from the cannula model. 

RESULTS 



Fig. 2 Model-based poses of the active cannula robot: 3D view 
(left) and view along the needle guide axis (right). 

The mean and maximum tip errors for the 28 cannula 
poses were 1.18 mm and 2.40 mm, respectively. The 
fiducial registration error was 0.278 mm, and steady- 
state errors of the base joints were on the order of 0.050 
mm (translations) and 0.5 degrees (rotations). 


DISCUSSION 

Observing that the registration error and base joint 
tracking errors are 1 to 2 Orders of magnitude below the 
cannula tip errors, it is reasonable to assume the 
majority of tip error is due to other sources. One likely 
cause is inaccuracy in the initial mounting of the 
cannula tubes; this was done by hand using a ruler and 
pen. Maximum tip error occurred when the precurved 
tube was at maximum extension. This error likely arose 
from the fact that the curvature preset into the curved 
tube may not have been perfectly circular. The shape 
setting could be improved by use of a high-precision jig 
and oven to more precisely set the circular curve. 

Nonetheless, the presently observed tip errors are 
quite small and approach the voxel size for MRI 
Scanners (about lxlxl mm ), and importantly we also 
note that real-time imaging will be available so that an 
image-based control loop can be closed on tip position 
during cannula insertion. Thus, the Overall accuracy of 
tip placement is not limited to the robot’ s free-space 
open loop tip accuracy, which is what is reported in this 
paper. Furthermore, we tested for MRI-compatibility in 
a Philips 3T Achieva Scanner, and the robot displayed 
no measurable effect on the signal-to-noise ratio of an 
fBIRN phantom. Image distortion tests with an ADNI 
phantom indicate a maximum distortion of 1.3 mm. 

These results demonstrate that MRI-guided 
pneumatic actuation is a promising solution for the 
minimally invasive treatment of epilepsy and of other 
intracranial diseases that either require or can benefit 
from MR imaging and thermometry. Future work 
includes Integration of the robotic platform with the 
MRI Scanner. Scanner experiments will focus on 
providing real-time feedback of tip location as well as 
using MR thermometry for real-time thermal dosimetry. 
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INTRODUCTION 

Magnetic manipulation of capsule endoscopes has the 
potential to make current gastrointestinal screening 
procedures faster, safer, and less invasive. To date, two 
electromagnetic Systems have been developed with the 
ability to perform five-degree-of-freedom (5-DOF) 
manipulation of an untethered magnetic device such as a 
magnetic capsule endoscope: the OctoMag System 
consists of eight electromagnets arranged around a 
hemisphere directed toward the manipulation workspace 
[1]; a System has been developed by Siemens, 
consisting of 12 electromagnets through which a patient 
is positioned, for the control of a capsule endoscope in a 
water-filled stomach [2]. Permanent-magnet actuation 
Systems are gaining attention for their ability to generate 
fields with clinically relevant strengths, inexpensively 
and in a compact form-factor, compared to 
electromagnetic Systems. Previous permanent-magnet 
Systems for capsule endoscopy have been limited to 
dragging and rolling capsule endoscope devices on the 
stomach’ s surface or in the large colon [3-5]. 

This proof-of-concept paper demonstrates magnetic 
3-DOF position and 2-DOF Orientation control of a 
mockup stomach capsule endoscope in fluid, using a 
single permanent magnet. 5-DOF manipulation of 
untethered devices has been previously demonstrated 
only with electromagnet Systems. 

MATERIALS AND METHODS 

The mockup capsule is actuated in a tank of water by an 
axially magnetized cylindrical NdFeB magnet with 
dipole moment MGI 3 (Grade N42, ||M|| = 26.2 A • 
m 2 ), positioned by a Yaskawa Motoman MH5 6-DOF 
robotic manipulator. The capsule contains a cube 
NdFeB magnet with its dipole moment m E IR 3 (Grade 
N52, || m || = 0.126 A • m 2 ) arranged parallel to the 
capsule ’s principal axis; the remainder of the capsule ’s 
volume is filled with air. The capsule ’s weight is 0.0153 
N and the buoyancy force in water is 0.0148 N. The 
capsule ’s 3-DOF position is triangulated by two Basler 
A602FC cameras. The actuator magnet, capsule, and 
experimental Setup are shown in Fig. 1. 

A magnetic torque x m = Mo m X H and magnetic 
force f m = i u 0 (m • V)H are applied to the center of the 
capsule ’s magnet by the magnetic field H E IR 3 
generated by the actuator magnet Q u 0 is the permeability 

This material is based upon work supported by the National 
Science Foundation under Grant No. 0952718. 
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Fig- 1 The actuator magnet (a) is used to control the mockup 
capsule endoscope (b) in a water-filled tank, using the robotic 
manipulator and vision System shown in (c). 


of free- space). The field produced by the actuator 
magnet is approximated by the point-dipole model: 

H = 3SW 0WiT -' )M - 0) 

where p E IR 3 is the vector from the actuator magnet’ s 
center to the capsule magnet’ s center, / E IR 3x3 is the 
identity matrix, and ~ denotes scaling to unit length. 

The capsule ’s position and heading are controlled by 
applying magnetic force and torque, respectively. We 
assume that the magnetic torque x m instantaneously 
aligns the capsule dipole moment m (and the capsule ’s 
heading) with the applied magnetic field H, and the 
capsule ’s heading can be controlled by adjusting the 
field direction H without Controlling the magnetic torque 
x m directly. The magnetic field (1) varies with the 
relative position p and the direction of the actuator 
dipole moment M . The total force f applied to the 
capsule consists of forces due to weight and buoyancy, 
which are both constant, and the magnetic force f m , 
which varies with p and M. In general, magnetic force 
also varies with m, however, we assume m to be aligned 
with H. The position p and actuator moment direction M 
(controlled by the robot manipulator) are related to field 
direction H and total force f by the nonlinear actuation 
function: 



Rather than explicitly inverting (2) to determine the 
necessary moment M and position p of the actuator 
magnet, given the desired field direction H and total 
applied force f, the Jacobian matrix E IR 6x6 is 
computed analytically, using the measured capsule 
position, and inverted using the Moore-Penrose 
pseudoinverse. The result relates small changes in the 
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(a) (b) 



Fig.2 The mockup capsule can be holonomically controlled with five degrees-of-freedom. In (a), the capsule is controlled to adjust 
its heading while remaining stationary in space. In (b), the capsule follows a U-shaped trajectory while maintaining a constant 
heading. The path taken by the actuator magnet is shown in both examples. Complete video demonstrations of (a) and (b) can be 
found at http://www.telerobotics.utah.edu/index.php/Research/PermanentMagnet. 


desired field direction and force to small changes in the 
actuator moment and position: 

AM] = t [AH 

_a p J jA Ufr 


( 3 ) 


which governs how the robotic manipulator moves the 
actuator magnet in space as the desired field heading 
(i.e., capsule heading) and applied force vary. The 
matrix ] ^ is always rank five, meaning that the capsule 
can be controlled with five degrees-of-freedom. The 
single uncontrollable DOF corresponds to capsule 
rotation about its magnetic dipole moment m. 

In practice, values of Af or AH and certain System 
configurations can produce large changes in Ap or AM, 
which may be unachievable by the robot manipulator in 
the worst case. The manipulator’ s motion can be 
reduced by temporarily sacrificing control over the 
capsule ’s heading, without losing control over its 
Position. This is performed by solving a constrained, 
weighted, least-squares minimization problem. 


DISCUSSION 

A feedback Controller, using the triangulated capsule 
Position obtained from the vision System, is used to 
servo the capsule to any desired position in the 
workspace. At every iteration, the Controller produces a 
desired change in force Af, and the desired change in 
heading AH is obtained from the user, which are 
converted into motion of the robotic manipulator by 
equation (3). An estimate of the capsule’s heading is 
obtained from the measured position using (1), and is 
controlled in an open-loop fashion (i.e., the capsule’s 
heading is not measured). 

Fig. 2 shows two capsule maneuvers that prior 
actuation Systems using a single permanent magnet are 
unable to perform. In Fig. 2(a), the capsule’s heading is 
rotated from a down-pointing to a side-pointing 
configuration, while the feedback Controller regulates 
position with no extemal contact. Nonintuitively, the 
actuator magnet does not remain directly above the 
capsule during the transition. In Fig. 2(b), the capsule’s 
position follows a U-shaped trajectory while its heading 
remains constant. These examples demonstrate simple 
maneuvers. Since the System is capable of 5 -DOF 
control, more complicated maneuvers are also possible. 


DISCUSSION 

The control scheme presented herein relies upon the 
capsule’s net weight (i.e., weight minus buoyancy) to 
apply downward forces in the direction of gravity. 
When the capsule is desired to move downward, the 
applied magnetic force is adjusted to be less than the net 
weight by repositioning the actuator magnet according 
to (3). The maximum downward force that can be 
applied (without placing the actuator below the capsule) 
is the capsule’s net weight. 

When the magnetic force balances the net weight, 
then the capsule levitates at equilibrium and is stabilized 
by the feedback Controller. The equilibrium distance 
depends on the capsule’s net weight and the strength of 
the capsule and actuator magnets, ||m|| and ||M||, 
respectively. If the capsule’s net weight is increased, a 
larger actuator magnet can be used with the same 
results. The closer the capsule is to neutral buoyancy, 
the smaller the magnetic force needed for levitation. 
However, since the capsule’s net weight is required to 
apply downward force, a neutrally buoyant capsule is 
undesired. 

In a clinical setting, the vision System used for 
capsule localization can be replaced by more 
appropriate localization methods such as RF 
triangulation, ultrasound, magnetic tracking, and other 
(potentially hybrid) approaches. 

REFERENCES 

[1] M. P. Kummer et al. OctoMag: An electromagnetic 
System for 5 -DOF wireless micromanipulation. IEEE 
Trans. Robot., 26(6) : 1 006 — 1017, 2010. 

[2] H. Keller et al. Method for navigation and control of a 
magnetically guided capsule endoscope in the human 
stomach. In Proc. IEEE Int. Conf. on Biomedical 
Robotics and Biomechatronics , pages 859—865, 2012. 

[3] S. Yim and M. Sitti. Design and rolling locomotion of a 
magnetically actuated soft capsule endoscope. IEEE 
Trans. Robot., 28(1): 183— 194, 2012. 

[4] G. Lien et al. Magnetic control System targeted for 
capsule endoscopic operations in the stomach: Design, 
fabrication, and in vitro and ex vivo evaluations. IEEE 
Trans. Biomed. Eng., 59(7):2068— 2079, 2012. 

[5] G. Ciuti et al. Robotic magnetic steering and locomotion 
of capsule endoscope for diagnostic and surgical 
endoluminal procedures. Robotica, 28(2): 199— 207, 
2010. 


The Hamlyn Symposium on Medical Robotics (2013) 


115 





Endoscopic Submucosal Dissection for Gastric Lesions using a Flexible 

Snake Robot - Early Assessment and Feasibility Study 

N.K. Patel 1 , T.P. Cundy 1 , J. Shang 1 , C. Payne\ C. Seneci 1 , V. Vitiello 1 , 

J. Clark 1 , J. Teare 1 , A. Darzi 1 , G.-Z. Yang 1 

The Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, 

Imperial College London 
nisha.patel2@imperial. ac. uk 


INTRODUCTION 

Annually, there are approximately one million cases of 
gastric cancer diagnosed globally. Whilst surgery has 
traditionally been the definitive treatment, flexible 
endoscopic techniques such as endoscopic submucosal 
dissection (ESD) are emerging as minimally invasive 
alternatives [1]. 

Thus far, the flexible endoscope has been adopted as 
the preferred instrument for ESD. However, this 
platform poses significant challenges to performing 
complex endoluminal surgery. Maintaining a target 
lesion in constant view for accurate dissection is 
technically and ergonomically demanding. In addition, 
triangulation of ESD tools is highly restricted and off- 
axis movements often prove problematic. Tissue 
traction and manipulation are also inadequate. These 
factors contribute to lengthy procedure times, 
significant operator bürden and complication rates. The 
purpose of this study is to explore the feasibility of a 
flexible snake robot as a technological solution for ESD 
of gastric lesions. 

METHODS 

The multi-articulated snake-like robot, based upon a 
universal-joint mechanism with embedded micromotor 
actuation [2], is 12.7mm in diameter and 150mm in 
length with a 50mm semi-rigid neck. Four distal joints 
permit eight degrees of freedom for kinematic 
configuration approaching multi-axis retroflexion 
capability [2]. Position-control of the motorised joints 
allow precise motions of the robot to be made. 

A camera and light source occupy 2.8mm and 
2.4mm internal channels respectively. Flexible 
endoscopic tools may be interchangeably deployed 
down a 3mm central working channel. 

Two porcine upper gastrointestinal specimens were 
placed into a validated endoscopic surgery Simulator 
tray (EASIE-R, ENDOSIM). The duodenum was 
clamped to allow gastric insufflation and simulated 
gastric lesions in the antrum and gastric body were 
created using submucosal normal saline blebs. 

RESULTS 

Controlled navigation was performed within the gastric 
cardia, fundus and greater curve using an ergonomic 
interface. Ten (10-20mm) lesions in the gastric body 
were excised along planned dissection margins. The 


lesions were marked circumferentially (soft coagulation 
mode, 50W) and a submucosal fluid cushion under the 
lesion was created using a HybridKnife integrated 
water-jet electrocautery tool (ERBE JET® 2 console, 
effect 30). A mucosal incision was made peripherally 
around the target lesion (EndoCut-I, effect 2, cut 
duration 3, interval 2). A HybridKnife, T-type (ERBE) 
was then used for dissection (forced coagulation mode, 
45 W, effect 3) (Figure 1). No perforations were found 
on close inspection. 

CONCLUSION 

This ex vivo feasibility study shows promising results 
supporting the role of the snake robot to navigate 
through the upper gastrointestinal tract and perform 
gastric ESD. Further in vivo studies are now required to 
optimise the robotic device and to establish optimal 
settings for ESD. This flexible snake platform offers 
enhanced stability, controlability, enhanced ergonomics 
and precision. It may overcome limitations presented 
by the flexible endoscope, thereby enabling the 
increased global uptake of ESD. 




Fig.l Ex vivo robotic ESD of gastric lesion (view taken from 
a flap created in the gastric body) a) marking b) submucosal 
cushion formation c) mucosal incision d) submucosal 
dissection 
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INTRODUCTION 


Arthrography is the evaluation of joint condition using 
imaging modalities such as computed tomography (CT) 
and magnetic resonance imaging (MRI). Among 
American children ages 5-14, there is on average over 
one sports-related injury per year, with a significant 
portion involving internal derangements of shoulders, 
hips, wrists, and other joints [1]. When there is concern 
for derangement of articular labral structures, as may be 
seen with these injuries or untreated congenital joint 
dysplasias, magnetic resonance (MR) arthrography is 
the modality of choice for evaluation of the labrum, 
articular cartilage, and other internal structures of the 
joint in children. Currently, this test requires two 
separate stages, an intra- articular contrast injection 
guided by fluoroscopy or ultrasound followed by an 
MRI. The inability to leverage the imaging capabilities 
of the MRI itself and the manual nature of needle 
placement lead to increased cost, anxiety, and in some 
cases prolonged sedation time, especially for the 
youngest and most anxious patients. Therefore, our 
ultimate goal is to develop an MRI-compatible shoulder 
arthrography robot. As a first step, we have developed a 
CT compatible prototype. 

In typical manual interventions, the physician guides 
the needle using cross-sectional images to reach the 
desired position in joint space or near the bone. 
Traditional needle manipulation often requires multiple 
passes to reach the target. In MRI guided interventions, 
patient access can be difficult, especially for closed 
bored Scanners. However, compared to CT imaging, 
MRI has advantages including: 1) no radiation exposure 
which is especially important for pediatrics; and 2) 
higher soft tissue contrast. Therefore, we are developing 
a novel patient-mounted robotic System to provide 
better targeting ability, improve the clinical workflow, 
and allow better access with the MRI Scanner bore. 

Many researchers have reported related work in the 
field of MRI and CT guided interventions. The high 
magnetic field present in the MRI environment is the 
major challenge to develop MRI compatible equipment. 
Nonmagnetic materials, MRI compatible actuators 
(piezo motors, pneumatic and hydraulic actuators), 
optical Tibers, encoders, and sensors are key elements of 
this kind of robotic System. Walsh et al. [2] developed a 
patient mounted robot called Robopsy. This robot is a 
telerobotic System with 2 degree-of-freedom (DOF) for 
needle guidance and insertion System. The robot is 
attached to the patient via an adhesive pad and optional 


strap points, so the device moves passively with patient 
motion and is thus inherently safe. This robot can Orient 
the needle about 2 axes and insert the needle 
automatically through j oystick control of the physician. 

Maurin et al. [3] developed a patient-mounted robot 
from a 5 DOF parallel structure with a semispherical 
workspace, particularly well suited to CT-based 
interventional procedures. A new robotic architecture 
was developed to perform interventional CT/MR 
procedures using both ultrasonic motors and pneumatics 
to position the needle and then insert it progressively. 
The whole robot can be sterilized and its mechanical 
error is less than 5mm [4]. 

Bricault et al. [5] developed a light robot with 5 
DOFs to perform abdominal and thoracic punctures 
under CT or MRI guidance for diagnostic or therapeutic 
purposes. The translation accuracy is 1mm. Rotation 
and inclination accuracy are less than 1°. This robot is a 
lightweight (1 kg) and compact (15 x 23 cm). 
Compressed air is used as energy source. 

In this paper we describe a small and lightweight 
patient-mounted CT and MRI robot under development 
for shoulder and hip arthrography for pediatric 
interventions. The robot has 3 DOFs and we plan to use 
piezo motors for actuation of the robot under MR 
imaging. A combination of adhesive tape and straps are 
planned for stable attachment of the robot to the patient 
body. Furthermore, to address the sterilization issues, 
the robot was designed to allow coverage of major 
portions of the robot by using sterile plastic sheet. 

MATERIALS AND METHODS 

As shown in Fig. 1, the concept of a four-link parallel 
mechanism with a spherical joint is used to develop the 
robot which gives us 2 rotational DOF about the 
spherical joint. The whole four-link parallel mechanism 
slides translationally through link 4 to add the third 
DOF. 



Fig* 1 schematic of the parallel robot 


The Hamlyn Symposium on Medical Robotics (2013) 


117 


As shown in the Solidworks model in Fig. 2(a), 
motor 1 and motor 2 rotate the needle with respect to 
the spherical joint to provide the 2 rotational DOFs. 
Motor 3 provides translational motion along with link 4 
through a timing beit and a pair of pulleys. Motor 4 is 
embedded in the robot to eventually drive the needle 
automatically by a remote command instead of manual 
needle insertion. 

For medical robotic Systems, sterilization is a critical 
issue that must be addressed. We considered three 
possibilities: 1) make the robot disposable, 2) use 
special material and sealing to make the robot 
sterilizable, 3) cover the robot with an appropriate 
protective sheet to avoid exposure of the robot to the 
environment. None of these methods alone seemed 
optimal for our application. Therefore, we combined 
these three methods and modified the first design to 
make the sterilization procedure easy, fast and cheap. 

As shown in Fig. 2(b), by separating link 4 into three 
parts and link 3 into two parts it is possible to cover 
most parts of the robot and then assemble pieces of the 
links 3 and 4 together. The parts that are shown with 
green lines can be either disposable or sterilizable. 



(a) 



(b) 


Fig. 2 Solidworks 3D model of the final design (a) without 
considering sterilization and (b) considering sterilization. 


As for control of the robot, we envision two 
possibilities: 1) a master-slave configuration, in which 
the physician can control the robot from the MRI 
control room with a Standard j oystick or with a MRI 
compatible Joystick from the MRI procedure room; and 
2) a positioning mode, in which the robot will pre-align 
its trajectory to some commanded path on the MRI 
images and an indication from the physician of the 
target point. In the second mode, the physician would 
then manually drive the needle to the target while the 
robot would maintain the desired trajectory. 


The robotic mechanism will be interfaced to a 
Computer through a Galil motor Controller. The Software 
will be implemented as a dialog-based MFC application 
in Visual C++ on the Windows operating System. The 
input from the user interface devices will be converted 
into control signals by the interfacing Software to drive 
the robotic mechanism. 

CLINICAL WORKFLOW 

Fig. 3 shows a prototype robot mechanism created from 
the Solidworks model using an Objet 500 rapid 
prototyping machine. This figure also shows how the 
needle is held. A protective sheet is used for separating 
the disposable parts from the rest of the robot. 



Fig. 3 First prototype of the robot with ABS material mounted 
on a body phantom. A protective cover is used to separate 
disposable part from the rest of the robot. 

DISCUSSION 

We have completed our initial design and prototype 
mechanical System of a robot for shoulder arthrography. 
We first plan to demonstrate the workflow and targeting 
capability in the CT environment with the System 
described here. The next Step will be to develop an MRI 
compatible Version. The ultimate goal is for the robot to 
be capable of automatically driving a needle and 
injecting contrast material during a real-time MRI 
sequence to observe the joint function and make an 
accurate diagnosis. 
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INTRODUCTION 

Since its inception thirty years ago, Transanal 
Endoscopic Microsurgery (‘TEMS’) has been 
performed under 3D vision from the TEMS stereoscope 
(R. Wolf). Subsequent alternative Systems, such as the 
TEO (Karl Storz) and TAMIS (Covidien) offer only 2D 
visualisation. 3D optics are said to improve the 
surgeon’s depth perception, assisting in the accurate 
performance of awkward procedures such as this [1]. 
However, the appeal of the 3D visualisation is 
compromised by the fact that it is obtained by viewing 
directly into the stereoscope, necessitating an 
uncomfortable operating posture. To address this, and to 
permit future clinical work on intraoperative image 
overlay [2], a novel 3D stereoscopic viewer was 
developed to meet the requirements of 3D vision with 
improved ergonomic configuration and was employed 
in two clinical cases. 



Fig. 1 CAD design, stereoscopic viewer unmounted and 
suspended from stand. 


METHODS 

Figure 1 shows the stereoscopic viewer, which was 
designed following the principles of the Wheatstone 
viewer [3]. Paired mirrors, lenses and video monitors 
(5.6” TVLogic VFM-056W 1280x800), receive input 
from the TEMS stereoscope via a pair of Storz HD 
cameras. The components of the viewer were mounted 
on a base plate, suspended from a custom-built frame 
(Gallops UK). 

Consecutive patients listed for TEMS were recruited 
into this feasibility study, approved by the regional 
ethics review panel (reference 07/Q0703/24): an 81 


year old female with a posterior TI rectal 
adenocarcinoma and a 73 year old male with a 
posterolateral tubullovillous adenoma. The stereoscopic 
viewer and paired camera heads were bagged and 
draped for sterility and connected to the TEM 
stereoscope (figure 2). 

Data recorded were: time taken 

for stereoscope setup and TEMS procedure. NASA- 
TLX and custom-designed usability questionnaires were 
completed post-operatively by the surgeon. Clinical 
follow-up of each case was undertaken. 



Fig. 2 Sterile draping of stereoscopic viewer and cameras, 
surgeon performing TEMS using the viewer. 


RESULTS 

Mean time to set up the stereoscopic viewer was 20 
minutes (performed concurrently with scrub trolley 
preparation). Use of the stereoscopic viewer was found 
to be feasible in both cases (figure 2) and both tumours 
were successfully removed without event. Mean 
operative time was 63 minutes. Mean NASA-TLX 
score was 41.5/120 (low being good) and custom- 
questionnaire on usability, 80%. Specific feedback 
from the surgeon included ‘high quality of 3D image’ 
and ‘improved comfort of head position’. Suggestions 
for improvement included reduced footprint of the 
Suspension frame and relocation of video monitor 
cables. Both patients made uneventful recoveries with 
no peri-operative complications. 

CONCLUSIONS 

This novel 3D stereoscopic viewer has been shown to 
be an effective means of visualisation in TEMS and 
early results from these 2 cases suggest improved 
ergonomics. Construction of the viewer cost under 
£2000 making it a simple and cost-effective solution, 
which is ready for more widespread clinical deployment 
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in TEMS and other procedures. Utilisation of such a 
viewer also paves the way for 3D intraoperative image 
augmentation in TEMS. 
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INTRODUCTION 

Successful and safe laparoscopic surgery is heavily 
dependent on its vision System, with the primary 
emphasis on the viewing angle and image stability [1]. 
Laparoscopic cameras, such as the Storz 10 mm 
laparoscope, are typically long, rigid tubes which have 
poor accessbility to target areas and require a second 
surgeon to operate. With a shifting enthusiam for 
natural orifice translumenal endoscopic surgery 
(NOTES), most groups prefer flexible endoscopes over 
the traditional rigid laparoscopes. However, these 
endoscopes were originally designed for intralumenal 
use, and tend to be application specific [1]. Thus, there 
is a technological and clinical need for a small, flexible 
camera designed for NOTES and other minimally 
invasive surgical procedures. It has been suggested that 
camera Systems for NOTES also be deployable, as 
flexible endoscopes occupy port space, hindering the 
use for additional tools [i]. These tools, like the 
Olympus GIF TYPE 160, can be 8.6 mm in diameter, 
occupying over half of a 12 mm port. Deployable 
camera Systems such as Pillcam, are unable to provide 
real time control due to their wireless Connectivity. 

The Core-Snake aims to bridge these gaps. The Core- 
Snake is a 10 mm diameter robot which can alter its 
body stiffness from being flexible to rigid via granulär 
jamming. Most commonly seen in vacuum packed bags 
of rice or coffee, granulär jamming is a phenomenon 
where a multitude of particles normally act like a fluid, 
but lock into a solid-like state when an external stress is 
applied [2], [3]. Thus, the Core-Snake is naturally 
compliant and can be pushed into position by the 
surgeon’ s laparoscopic tools, then it can lock its current 
Position when a differential between the internal and 
external pressure is applied. In this case, the differential 
is induced by vacuuming the interior of the snake. This 
variable stiffness mechanism not only provides the 
surgeon with a wide viewing angle and accessibility 
when the snake is flexible, but also a stable vision 
platform when the snake is rigid [4]. Though it requires 
a tether for the video cables and pneumatic tubing, the 
cable and tube bündle is only 4 mm in diameter. The 
design of the robot allows the snake to be deployed into 
the body cavity, thus only occupying 4 mm of port 
space after initial insertion, a significant improvement 
over current endoscopes. 



Fig. 1 The Core-Snake, a low cost, granulär jamming-based 
flexible laparaoscopic camera. Shown here its ability to access 
difficult areas, while occupying 4 mm of trocar port space. 

MATERIALS AND METHODS 

The Core-Snake, seen in Fig. 1 and 2, is 10 mm in 
diameter, composed of three, 65 mm long segments 
filled with 1.5 mm diameter plastic spherical granules, 
and followed by a 4 mm tether. The outer membrane is 
made with a 0.12 mm thick PVC film. The section 
dividers are ABS plastic and were printed with a rapid 
prototyping machine. The tip camera is a 10 mm 
diameter CMOS sensor with a 640x480 resolution at 30 
frames per second and was connected to a Computer via 
USB. The tip also includes 4 white LEDs for light. The 
pneumatic line is a 2 mm outer diameter silicone rubber 
tube. Vacuum pressure is achieved with a Mastercool 
90066-2 V-220 pump, and measured by a Honeywell 0- 
30 PSI absolute pressure sensor. 



Pneumatic tube Camera 1 0 mm 

I 1 


Fig. 2 Here, jamming the Core-Snake granules locks it into its 
rigid state, and holding a 180 degree bend. The flexibility and 
variable stiffness of the camera System allows surgeons to 
navigate to a area of interest and lock the camera in the 
Position. 

To test the stiffness ränge of the robot, bending tests 
were conducted on one segment of the snake. The end 
of the segment was fixed, and the tip was deflected by 
10 mm. The required load for this deflection was 
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measured by an ATI Nanol7 Force/Torque sensor. The 
force and deflection measurements were then converted 
to Young’s modulus E for a comparison of stiffness. 

Extemal groups utilizing granulär jamming, particularly 
for snake-like or elephant trunk-like robots have shown 
that ground coffee is the ideal granule type for jamming 
[5]. However, in medical applications, concerns for 
using organic materials in a laparoscopic camera System 
resulted in tests to examine if there is a plastic 
substitute. The synthetic substitute would need to match 
the stiffness of ground coffee and withstand an 
autoclave, the most common method of disinfection. 


RESULTS 



Fig.3 The snake bending due to weight when soft (top) vs the 
snake holding a horizontal position when rigid (bottom). 



Fig. 5 The Core-Snake inside a Labcaire Autoscope ISIS 
Automated Endoscope Reprocessor. 


Bending tests, shown in Fig. 4, performed between 
ground coffee and plastic spheres for the rigid case 
showed that a synthetic substitute improved the 
stiffness. Unlike coffee particles, the plastic granules are 
able to withstand the sterilization process via steaming, 
both validating the use of granulär jamming in a 
medical de vice and the use of synthetic granules. 

DISCUSSION 

This paper presents a new type of NOTES-orientated 
laparoscopic camera System which is variable stiffness, 
deployable, and low cost. The total cost of the snake, 
excluding the vacuum pump, is approximately 20 GBP, 
50 times less than traditional laparoscopes. Though 
some preliminary sterilization tests have been 
performed for the granules, they have not been done for 
the manipulator as a unit. However, because of the low 
cost nature of the System, an evaluative study should be 
done on the practicality of a disposable camera System. 
Currently, the positioning of the camera can be done by 
the surgeon’s laparoscopic tools, reducing the need for a 
“cameraman.” Future work involve adding a high 
defmition camera, increasing the stiffness ränge of the 
snake, adding an integrated actuation System, and 
testing it in- vivo. 


The bending test results show that, after deflected, one 
segment can exhibit 0.20 N of force at 101 kPa (soft 
state) and 0.42 N at 20 kPa (rigid state). The stiffness, 
Young’s modulus E, is calculated from the bending 
results by E = (4FL 3 )/(37tdr 4 ), where F is the loading 
force, L is the segment length, d is the deflection 
distance, and r is the segment radius. Thus, E = 3.73 and 
7.83 MPa for the soft and rigid States, respectively. 
These preliminary results show that the Core-Snake is 
able to increase its stiffness by 2.1 times. Fig. 3 visually 
shows what these stiffness values represent: the varied 
ability to hold the Core-Snake in a horizontal position. 
In other words, the rigidified snake is a stable platform. 
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Fig. 4 Ground coffee (left) vs plastic spheres (right) in the 
bending test when jammed. Plastic spheres outperformed 
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